Papers with quality
Extremely efficient online query encoding for dense retrieval (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing dense retrieval systems use the same model architecture for encoding both passages and queries, even though queries are much shorter and simpler than passages. |
| Approach: | They propose a small efficient RNN query encoder that can reduce latency by 12 with only a minor decrease in quality. |
| Outcome: | The proposed solution reduces latency by up to 12 while achieving 35.5 MRR@10 score. |
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets (2022.tacl-1)
Copied to clipboard
Julia Kreutzer, Isaac Caswell, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii-Orshikh, Allahsera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, Monang Setyawan, Supheakmungkol Sarin, Sokhar Samb, Benoît Sagot, Clara Rivera, Annette Rios, Isabel Papadimitriou, Salomey Osei, Pedro Ortiz Suarez, Iroro Orife, Kelechi Ogueji, Andre Niyongabo Rubungo, Toan Q. Nguyen, Mathias Müller, André Müller, Shamsuddeen Hassan Muhammad, Nanda Muhammad, Ayanda Mnyakeni, Jamshidbek Mirzakhalov, Tapiwanashe Matangira, Colin Leong, Nze Lawson, Sneha Kudugunta, Yacine Jernite, Mathias Jenny, Orhan Firat, Bonaventure F. P. Dossou, Sakhile Dlamini, Nisansa de Silva, Sakine Çabuk Ballı, Stella Biderman, Alessia Battisti, Ahmed Baruwa, Ankur Bapna, Pallavi Baljekar, Israel Abebe Azime, Ayodele Awokoya, Duygu Ataman, Orevaoghene Ahia, Oghenefego Ahia, Sweta Agrawal, Mofetoluwa Adeyemi
| Challenge: | Lower-resource corpora have systematic issues, including mislabeled or nonstandard/ambiguous language codes. |
| Approach: | They manually audit the quality of 205 language-specific corpora released with five major public datasets. |
| Outcome: | The results show that lower-resource corpora have systematic issues even for non-proficient speakers. |
Multi-Programming Language Sandbox for LLMs (2025.acl-demo)
Copied to clipboard
Shihan Dou, Jiazheng Zhang, Jianxiang Zang, Yunbo Tao, Weikang Zhou, Haoxiang Jia, Shichun Liu, Yuming Yang, Shenxi Wu, Zhiheng Xi, Muling Wu, Rui Zheng, Changze Lv, Limao Xiong, Shaoqing Zhang, Lin Zhang, Wenyu Zhan, Rongxiang Weng, Jingang Wang, Xunliang Cai, Yueming Wu, Ming Wen, Yixin Cao, Tao Gui, Xipeng Qiu, Qi Zhang, Xuanjing Huang
| Challenge: | MPLSandbox is an out-of-the-box multi-programming language sandbox designed to provide unified and comprehensive feedback from compiler and analysis tools for Large Language Models (LLMs). |
| Approach: | They propose a multi-programming language sandbox that provides unified feedback from compilers and analysis tools for Large Language Models. |
| Outcome: | The proposed multi-language sandbox can provide comprehensive feedback from compilers and analysis tools for large language models (LLMs). |
Locally Typical Sampling (2023.tacl-1)
Copied to clipboard
| Challenge: | a discrepancy in probabilistic language generators has puzzled the language generation community for years . |
| Approach: | They propose a method for local typical sampling to enforce a conditional entropy criterion for probabilistic models. |
| Outcome: | The proposed method can provide new insights into why high-probability texts can be dull or repetitive. |
EffEval: A Comprehensive Evaluation of Efficiency for MT Evaluation Metrics (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent surge of interest in developing evaluation metrics based on pretrained large language models (LLMs) can better cope with lexical variation. |
| Approach: | They propose to replace computation-intensive transformers with lighter alternatives and employ linear and quadratic approximations for alignment algorithms on top of LLM representations. |
| Outcome: | The proposed approach replaces computation-intensive transformers with lighter alternatives and employs linear and quadratic approximations for alignment algorithms on top of LLM representations. |
QiuNiu: A Chinese Lyrics Generation System with Passage-Level Input (2022.acl-demo)
Copied to clipboard
| Challenge: | Existing systems based on attributes or keywords render lyrics generation very limited . previous studies focused on generating lyrics based only on attributes and keywords . |
| Approach: | They propose to use Chinese passage-level text as input for lyrics generation . they initialize parameters with custom pretrained Chinese GPT-2 model and adopt a two-step process to fine-tune the model for better alignment between passage- level text and lyrics. |
| Outcome: | The proposed system is conditioned on passage-level text rather than attributes or keywords, rendering limited control over the content of the lyrics. |
Annotation Process for the Dialog Act Classification of a Taglish E-commerce Q&A Corpus (D19-51)
Copied to clipboard
| Challenge: | Existing studies on DA classification in general contexts have not addressed this problem. |
| Approach: | They constructed a text-based corpus of 7,265 posts from the question and answer section of products on Lazada Philippines. |
| Outcome: | The text-based corpus of 7,265 posts from the question and answer section of products on Lazada Philippines was constructed using a tagset for DA classification . the corpus was composed dominantly of single-label posts, with 34% of the corpuse having multiple intent tags. |
Towards Answer-unaware Conversational Question Generation (D19-58)
Copied to clipboard
| Challenge: | Existing frameworks for conversational question generation are answeraware, but are not able to generate corresponding answers . a number of question generation methods are developed for text-based question answering . |
| Approach: | They propose a framework for conversational question generation that is unaware of the corresponding answers. |
| Outcome: | The proposed framework is effective but answeraware, the authors show . it improves quality of generated questions if question foci and question patterns are identified . |
Aspect-based Analysis of Advertising Appeals for Search Engine Advertising (2022.naacl-industry)
Copied to clipboard
| Challenge: | ad creators must consider various aspects of advertising appeals such as price, product features, and quality in their ac work. |
| Approach: | They propose to use a dataset of ad texts to explore the effective aspects of advertising appeals (A3) for different industries to assist a search engine ap creators. |
| Outcome: | The proposed model can detect aspects of ad texts and help them estimate their performance. |
Rad-Flamingo: A Multimodal Prompt driven Radiology Report Generation Framework with Patient-Centric Explanations (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing reports are labor-intensive and expert-intensive, resulting in inconsistencies and a lack of patient-centered insight. |
| Approach: | They propose a multimodal prompt-driven report generation framework that integrates diverse data modalities to produce comprehensive and context-aware radiology reports. |
| Outcome: | The proposed framework improves report quality, improves understandability and could foster better patient-doctor communication. |
Does Simultaneous Speech Translation need Simultaneous Models? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Simultaneous speech translation (SimulST) systems strive for high output quality but also low latency. |
| Approach: | They propose to train SimulST offline without additional training or adaptation . they also show offline training achieves similar or better quality compared to offline training . |
| Outcome: | The proposed model can serve both offline and simultaneous applications without additional training or adaptation. |
I-MCTS: Enhancing Agentic AutoML via Introspective Monte Carlo Tree Search (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing LLM-based agents struggle with low diversity and suboptimal code generation. |
| Approach: | They propose an approach that iteratively expands tree nodes through an introspective process that meticulously analyzes solutions and results from parent and sibling nodes. |
| Outcome: | The proposed approach shows a 4% improvement in performance compared to the strong open-source AutoML agents. |
Med-CoDE: Medical Critique based Disagreement Evaluation Framework (2025.naacl-srw)
Copied to clipboard
| Challenge: | Existing evaluation methods for large language models lack robustness and accuracy in medical contexts. |
| Approach: | They propose an evaluation framework for medical LLMs that measures disagreement between model-generated responses and established medical ground truths. |
| Outcome: | The proposed evaluation framework captures accuracy and reliability in medical settings. |
InsBank: Evolving Instruction Subset for Ongoing Alignment (2025.findings-emnlp)
Copied to clipboard
Jiayi Shi, Yiwei Li, Shaoxiong Feng, Peiwen Yuan, Xinglin Wang, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Huan Ren, Yao Hu, Kan Li
| Challenge: | Recent studies emphasize that quality and diversity of instruction data are more crucial than quantity, highlighting the need to select diverse, high-quality subsets to reduce training costs. |
| Approach: | They propose to use a continuously updated repository to integrate the latest valuable instruction data with a progressive evolution framework to evolve InsBank over time. |
| Outcome: | The proposed framework outperforms baselines in InsBank evolution and extracts budget-specific subsets. |
Information-Theoretic Probing with Minimum Description Length (2020.emnlp-main)
Copied to clipboard
| Challenge: | Despite widespread adoption of probes, differences in their accuracy fail to adequately reflect differences in representations. |
| Approach: | They propose an alternative to the standard probes, information-theoretic probing with minimum description length (MDL). |
| Outcome: | The proposed method agrees in results and is more informative and stable than the standard probes. |
A Community-Driven Data-to-Text Platform for Football Match Summaries (2024.lrec-main)
Copied to clipboard
| Challenge: | Prosebot is a community-driven data-to-text platform designed to generate textual summaries of football matches derived from match statistics. |
| Approach: | They describe the architecture and deployment of a community-driven data-to-text platform for generating textual summaries of football matches derived from match statistics. |
| Outcome: | The proposed system enhances visibility of lower-tier matches, traditionally accessible only through data tables. |
Parallel sentences mining with transfer learning in an unsupervised setting (2021.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods to mine parallel sentences in low-resource environments are not suitable for many low-level language pairs. |
| Approach: | They propose an approach based on transfer learning to mine parallel sentences in an unsupervised setting using bilingual corpora of low-resource language pairs. |
| Outcome: | The proposed model improves the performance of mined parallel sentences at two real-world low-resource language pairs compared with previous methods. |
CLIReval: Evaluating Machine Translation as a Cross-Lingual Information Retrieval Task (2020.acl-demos)
Copied to clipboard
| Challenge: | evaluating machine translation (MT) with cross-lingual information retrieval is relatively time-consuming and subjective. |
| Approach: | They propose a toolkit that evaluates machine translation with a proxy task of cross-lingual information retrieval. |
| Outcome: | The proposed toolkit is based on the "metrics shared task" of WMT2019. |
Language Scaling for Universal Suggested Replies Model (2021.naacl-industry)
Copied to clipboard
Qianlan Ying, Payal Bajaj, Budhaditya Deb, Yu Yang, Wei Wang, Bojia Lin, Milad Shokouhi, Xia Song, Yang Yang, Daxin Jiang
| Challenge: | We consider scaling automated suggested replies (SR) to multiple languages for a commercial email application. |
| Approach: | They propose a multi-lingual multi-task continual learning framework with auxiliary tasks and language adapters to train universal language representation across regions. |
| Outcome: | The proposed model reduces catastrophic forgetting and improves cross-lingual transfer across languages while reducing training costs. |
TranslateLocally: Blazing-fast translation running on the local CPU (2021.emnlp-demo)
Copied to clipboard
| Challenge: | Using cloud-based translation providers carries privacy risks, as users lose control of their data once it enters the web. |
| Approach: | They propose a desktop translation application that runs locally on a user's desktop or laptop CPU. translateLocally delivers cloud-like translation speed and quality even on 10 year old hardware. |
| Outcome: | The open-source translation system runs on Linux, Windows and macOS on desktops and laptops. |
Robust ASR Error Correction with Conservative Data Filtering (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Error correction (EC) based on large language models is an emerging technology to enhance the performance of automatic speech recognition systems. |
| Approach: | They propose to pair large set of ASR hypotheses with gold references to improve linguistic acceptability over sources and be inferable from available context. |
| Outcome: | The proposed approach significantly reduces overcorrection and improves quality in out-of-domain (OOD) settings. |
How Decoding Strategies Affect the Verifiability of Generated Text (2020.findings-emnlp)
Copied to clipboard
Luca Massarelli, Fabio Petroni, Aleksandra Piktus, Myle Ott, Tim Rocktäschel, Vassilis Plachouras, Fabrizio Silvestri, Sebastian Riedel
| Challenge: | Recent advances in pre-trained language models have generated text of an increasingly high quality. |
| Approach: | They propose a decoding strategy that produces less repetitive and more verifiable text. |
| Outcome: | The proposed method produces less repetitive and more verifiable text than previously used decoding strategies. |
Improving Top-K Decoding for Non-Autoregressive Semantic Parsing via Intent Conditioning (2022.coling-1)
Copied to clipboard
| Challenge: | Semantic parsing (SP) is a core component of modern virtual assistants like Google Assistant and Amazon Alexa. |
| Approach: | They propose a non-autoregressive (NAR) semantic parser that introduces intent conditioning on the decoder. |
| Outcome: | The proposed model reduces inference latency while maintaining competitive parsing quality. |
TWEETSUMM - A Dialog Summarization Dataset for Customer Service (2021.findings-emnlp)
Copied to clipboard
Guy Feigenblat, Chulaka Gunasekara, Benjamin Sznajder, Sachindra Joshi, David Konopnicki, Ranit Aharonov
| Challenge: | a dataset focused on customer care dialog summarization is the first to focus on real-world customer care conversations . it contains extractive and abstractive summaries, and extractive summarizing methods are also introduced . |
| Approach: | They present a customer care dialog summarization dataset with 6500 human annotated summaries . they introduce an unsupervised method for extracting dialog summary data . |
| Outcome: | The proposed method is based on real-world customer support dialogs and includes extractive and abstractive summaries. |
Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order (2020.acl-main)
Copied to clipboard
| Challenge: | Large-scale pretrained language models such as masked language model (MLM) have brought significant improvements to many NLU and NLG tasks. |
| Approach: | They propose a probabilistic masking scheme for the masked language model and a model with a uniform prior distribution on the masking ratio. |
| Outcome: | The proposed model outperforms BERT on a bunch of downstream NLG tasks. |
LLM-based Literal Example Generation for Japanese Multiword Expressions (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing work on Japanese multiword expressions has focused on detecting idiomatic usages in context, leaving literal readings underrepresented. |
| Approach: | They propose to use corpus non-literal usages as contrastive cues for controlled prompting . they compare their results to a test that compares model predictions with human judgments . |
| Outcome: | The proposed model provides more accurate literal examples than prompts that include no hints or literal information. |
Translation between Molecules and Natural Language (2022.emnlp-main)
Copied to clipboard
| Challenge: | MolT5 pretrains models on unlabeled natural language text and molecule strings . bringing a new drug to market can cost over a billion dollars and take over ten years . |
| Approach: | They propose a self-supervised learning framework for pretraining models on unlabeled natural language text and molecule strings. |
| Outcome: | The proposed framework pretrains models on unlabeled natural language text and molecule strings, and it generates high quality outputs. |
Songs Across Borders: Singable and Controllable Neural Lyric Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Obtaining singable lyric translations can facilitate the globalization of the music publishing industry . |
| Approach: | They formalize lyric translation into a constrained translation problem and instantiate them to an English-Chinese system. |
| Outcome: | The proposed model achieves 99.85%, 99.00%, and 95.52% on length accuracy, rhyme accuracy, and word boundary recall. |
Transformer and seq2seq model for Paraphrase Generation (D19-56)
Copied to clipboard
| Challenge: | Existing methods for generating paraphrases fall into one of these broad categories -rule-based, seq2seq, deep generative models and a varied combination. |
| Approach: | They propose a framework that combines transformer and sequence-to-sequence models for better quality of generated paraphrases. |
| Outcome: | The proposed framework improves on two datasets-QUORA and MSCOCO using transformer and sequence-to-sequence models. |
Automating Human Evaluation of Dialogue Systems (2022.naacl-srw)
Copied to clipboard
| Challenge: | a recent study shows that human evaluations of dialogue systems weakly reflect human judgments. |
| Approach: | They propose a BERT-based model that fine-tunes a model with three prediction heads to predict whether the system-generated output is natural, fluent, and informative. |
| Outcome: | The proposed model achieves an average accuracy of 77% over the 3 labels . it also uses three different models to compute the labels compared to three separate models . |
Optimizing Retrieval-Augmented Generation for E-Commerce How-To Assistance (2026.acl-industry)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have significantly improved Retrieval-Augmented Generation (RAG), enabling assistants that can reliably ground responses in external knowledge sources while maintaining high-quality natural language interaction. |
| Approach: | They propose a RAG-based How-To Assistant that groundes responses in a proprietary knowledge base to provide personalized customer support. |
| Outcome: | The proposed assistant can ground responses in a proprietary knowledge base while maintaining high-quality natural language interaction. |
Multimodal Generation with Consistency Transferring (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for multimodal content generation are limited to unimodal content production due to high training complexity, significant costs, and inadequate emphasis on model constraints. |
| Approach: | They propose a method to generate multimodal content with constraints on adjacent steps and a layer-based layer-constrained transfer between adjacent steps to improve denoising capabilities. |
| Outcome: | The proposed method improves the model’s ability to capture actions and depict backgrounds more effectively and improves video generation speed by approximately 40% and quality by about 39.3%. |
Reformatted Alignment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Current methods to improve data quality are labor-intensive or prone to factual errors caused by LLM hallucinations. |
| Approach: | They propose a method which reformats the responses of instruction data into a format that better aligns with pre-established criteria and the collated evidence. |
| Outcome: | The proposed approach minimizes human annotation, hallucination, and the difficulty in scaling, remaining orthogonal to existing alignment techniques. |
Ad Headline Generation using Self-Critical Masked Language Model (2021.naacl-industry)
Copied to clipboard
| Challenge: | We propose a programmatic solution to generate product advertising headlines using retail content. |
| Approach: | They propose a programmatic solution to generate product advertising headlines using retail content . they use Reinforcement Learning (RL) Policy gradient methods on Transformer . |
| Outcome: | The proposed method outperforms existing methods in overlap metrics and quality audits. |
MonoTODia: Translating Monologue Requests to Task-Oriented Dialogues (2025.naacl-industry)
Copied to clipboard
| Challenge: | Data scarcity is one of the main problems when it comes to real-world applications of transformer-based models. |
| Approach: | They propose a method to source annotated German monologues from existing monologue material to train TOD systems. |
| Outcome: | The proposed model can be used to train TOD systems on a real-world example of a travel booking service. |
An Exploration of Post-Editing Effectiveness in Text Summarization (2022.naacl-main)
Copied to clipboard
Vivian Lai, Alison Smith-Renner, Ke Zhang, Ruijia Cheng, Wenjuan Zhang, Joel Tetreault, Alejandro Jaimes-Larrarte
| Challenge: | Automated summarization methods are efficient but can suffer from low quality. |
| Approach: | They conducted an experiment with 72 participants to compare post-editing provided summaries with manual summarization for summary quality, human efficiency, and user experience. |
| Outcome: | The results show that post-editing improves summary quality, human efficiency, and user experience on formal (XSum news) and informal (Reddit posts) text. |
Automatic Correction of Human Translations (2022.naacl-main)
Copied to clipboard
| Challenge: | Despite recent advances in machine translation, a tremendous amount of translated content in the world is still written by humans. |
| Approach: | They propose a task of translation error correction (TEC) that corrects human-generated translations by correcting all errors in a source sentence and a human-created translation. |
| Outcome: | The proposed system improves translation accuracy by 5.1 points compared to MT systems with human errors . |
Fast Whitespace Correction with Encoder-Only Transformers (2023.acl-demo)
Copied to clipboard
| Challenge: | a new tool for whitespace correction is available for text with spurious spaces . the tool is 900 times faster than the previous best tool for text correction . |
| Approach: | They propose to combine whitespace correction with a character-level encoder-decoder model and a byte-level byte encoder only model to improve quality. |
| Outcome: | The proposed tool is over 900 times faster than the previous best tool, with the same high quality. |
GreaterPrompt: A Unified, Customizable, and High-Performing Open-Source Toolkit for Prompt Optimization (2025.acl-demo)
Copied to clipboard
| Challenge: | Recent advances in prompt optimization have introduced diverse techniques that automatically enhance prompts to better align model outputs with user expectations. |
| Approach: | They propose a framework that unifies different methods under a unified, customizable API while delivering highly effective prompts for different tasks. |
| Outcome: | The proposed framework unifies multiple methods under a unified, customizable API while delivering highly effective prompts for different tasks. |
ReportGPT: Human-in-the-loop Verifiable Table-to-Text Generation (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Recent advances in the quality and accessibility of large language models have precipitated a surge in user-facing tools for content generation. |
| Approach: | They propose a pipeline framework for verifiable human-in-the-loop table-to-text generation that is based on a domain specific language and a set of modules that use it as a representation for generating verifierable commentary. |
| Outcome: | The proposed framework learns from human feedback in real-time, needing only a few samples to improve performance. |
FAST: Fast Annotation tool for SmarT devices (2021.emnlp-demo)
Copied to clipboard
| Challenge: | In real-world applications, annotators with the same attributes are required to annotate whether the outputs of natural language generation systems are fluent or not. |
| Approach: | They propose an annotation tool for application tasks that focuses on the user experience of mobile devices and can be customized to fit various tasks. |
| Outcome: | The proposed tool can annotate faster than existing methods while maintaining the quality of annotation. |
Stretch-VST: Getting Flexible With Visual Stories (2021.acl-demo)
Copied to clipboard
| Challenge: | Existing visual storytelling models produce stories with fixed lengths of five sentences and the fix-length stories carry limited details and provide ambiguous textual information to the readers. |
| Approach: | They propose to “stretch” visual storytelling frameworks by adding appropriate knowledge to the model to generate long stories. |
| Outcome: | The proposed framework provides better focus and detail when long stories are generated without deteriorating the quality. |
MergeIT: From Selection to Merging for Efficient Instruction Tuning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for instruction tuning rely on LLMs to score instruction quality . existing methods rely only on Llms to rank instruction quality, but this approach is expensive and time-consuming . |
| Approach: | They propose a novel LLM-based Merging strategy for better Instruction Tuning that shifts the focus from selection to synthesis. |
| Outcome: | The proposed method reduces time and computational cost while preserving diversity and reducing redundancy. |
Kandinsky 3: Text-to-Image Synthesis for Multifunctional Generative Framework (2024.emnlp-demo)
Copied to clipboard
Arkhipkin Vladimir, Viacheslav Vasilev, Andrei Filatov, Igor Pavlov, Julia Agafonova, Nikolai Gerasimenko, Anna Averchenkova, Evelina Mironova, Bukashkin Anton, Konstantin Kulikov, Andrey Kuznetsov, Denis Dimitrov
| Challenge: | Text-to-image (T2I) diffusion models are popular for image manipulation, but also for video generation. |
| Approach: | They propose a novel T2I diffusion model based on latent diffusion that extends the base model for various applications. |
| Outcome: | The proposed model achieves high quality and photorealism and is 3 times faster than the base model. |
English Language Spelling Correction as an Information Retrieval Task Using Wikipedia Search Statistics (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing spelling correction tools lack training or annotated data sets to perform . many spelling correction utilities suffer due to the size and quality of dictionaries available to aid correction. |
| Approach: | They propose a dynamic spelling correction tool that uses the Wikipedia dataset search API to aid misspelled term identification and automatic replacement. |
| Outcome: | The proposed spelling correction tool performs comparable to the industry-standard spelling correction algorithm, Hunspell. |
Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are capable of understanding multi-modal content, but textonly human-computer interaction is not sufficient for many application scenarios. |
| Approach: | They propose a video-to-text generation task and a multi-modal framework that bootstraps cross-modal training from frozen pre-trained visual & audio encoders and frozen LLMs. |
| Outcome: | The proposed framework can understand both visual and auditory content in video and generate meaningful responses grounded in the visual and audio information presented in the videos. |
CrudeOilNews: An Annotated Crude Oil News Corpus for Event Extraction (2022.lrec-1)
Copied to clipboard
| Challenge: | a corpus of English crude oil news for event extraction is presented . the corpus contains 425 news articles with approximately 11k events annotated . |
| Approach: | They present a corpus of English Crude Oil news for event extraction . it is the first of its kind for Commodity News and contributes to text mining . |
| Outcome: | The proposed corpus of English crude oil news is the first of its kind for Commodity News . the annotated news articles are compared with the standard news articles . |
Auto-Cypher: Improving LLMs on Cypher generation via LLM-supervised generation-verification framework (2025.naacl-short)
Copied to clipboard
| Challenge: | Graph databases like Neo4j are gaining popularity for handling complex, interconnected data, over traditional relational databases. |
| Approach: | They propose an automated pipeline to generate Cypher queries for Neo4j using LLM-As-Database-Filler, a novel strategy for ensuring Cyphere query correctness. |
| Outcome: | The proposed pipeline generates high quality Cypher data containing 29.8k instances across various domains and queries with varying complexities. |
Leveraging Moment Injection for Enhanced Semi-supervised Natural Language Inference with Large Language Models (2025.naacl-short)
Copied to clipboard
| Challenge: | Existing studies have used class-specific fine-tuned large language models to generate hypotheses and assign pseudo-labels but discarded many LLM-constructed samples to ensure the quality. |
| Approach: | They propose to leverage LLM-constructed samples by injecting the moments of labeled samples during training to properly adjust the level of noise. |
| Outcome: | The proposed method outperforms strong baselines on multiple NLI datasets in low-resource settings. |
DiffusPoll: Conditional Text Diffusion Model for Poll Generation (2024.findings-acl)
Copied to clipboard
| Challenge: | Social media platforms manipulate public opinion through sheer numbers and cause biases, authors say . they say new paradigm for poll generation can generate high-quality samples while preserving diversity . |
| Approach: | They propose a non-autoregressive diffusion model that uses masks to generate polls . they use attribute tags to enhance the quality of polls and to diversify poll options . |
| Outcome: | The proposed model matches the Transformer model while offering greater diversity and quality. |
Evaluating Cost-Efficiency of LLMs in a RAG Setup on Polish Wikipedia: Quality vs. Energy Consumption (2026.eacl-srw)
Copied to clipboard
| Challenge: | Retrieval-augmented generation systems are a dominant paradigm for knowledge-intensive applications. |
| Approach: | They evaluate language models from 4B to 70B parameters within a Polish Wikipedia-based RAG pipeline. |
| Outcome: | The proposed model selection process reduces energy consumption by 83% and improves quality. |
Semantic Outlier Removal with Embedding Models and LLMs (2025.acl-industry)
Copied to clipboard
Eren Akbiyik, João F. M. De Almeida, Rik Melis, Ritu Sriram, Viviana Petrescu, Vilhjálmur Vilhjálmsson
| Challenge: | Modern text processing pipelines require robust methods to remove extraneous content while preserving a document’s core message. |
| Approach: | They propose a method that leverages multilingual sentence embeddings and approximate nearest-neighbor search to identify and excise unwanted text segments. |
| Outcome: | Experiments on HTML datasets show that SORE outperforms structural methods and yields high precision in diverse scenarios. |
LaMP-QA: A Benchmark for Personalized Long-form Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Personalization in question answering systems remains underexplored due to lack of resources . a new benchmark for personalized answer generation is being developed . |
| Approach: | They propose a LaMP-QA benchmark for evaluating personalized answer generation . they compare evaluation strategies for generating personalized responses and measure their alignment with human preferences. |
| Outcome: | The proposed benchmark shows that incorporating the personalized context provided leads to performance improvements. |
Diffusion Directed Acyclic Transformer for Non-Autoregressive Machine Translation (2025.acl-short)
Copied to clipboard
| Challenge: | Non-autoregressive transformers (NATs) often encounter performance challenges due to the multi-modality problem. |
| Approach: | They propose a direct-acyclic transformer (DAT) that captures multiple translation modalities to paths in a Directed Acyclic Graph (DAG) this allows the model to integrate latent variables into the model, which is crucial for DAT to achieve state-of-the-art performance. |
| Outcome: | The proposed model captures multiple translation modalities to paths in a Directed Acyclic Graph (DAG) but the collaboration with the latent variable introduced through the Glancing training is crucial for the model to attain state-of-the-art performance. |
Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for bilingual lexicon induction are linear and require simplifying assumptions. |
| Approach: | They propose methods that combine unsupervised bitext mining and unsupervised word alignment to produce higher quality lexicons. |
| Outcome: | The proposed method outperforms the state-of-the-art on the BUCC 2020 task by 14 F1 points . further analysis suggests they are comparable quality . |
In-Context Example Selection via Similarity Search Improves Low-Resource Machine Translation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have shown that in-context examples for machine translation are beneficial for high-resource languages. |
| Approach: | They propose to use in-context examples for machine translation (MT) they argue that similarity-based selection can improve MT . |
| Outcome: | The proposed approach improves machine translation (MT) and low-resource languages. |
NLG Evaluation Metrics Beyond Correlation Analysis: An Empirical Metric Preference Checklist (2023.acl-long)
Copied to clipboard
| Challenge: | a systematic review of automatic evaluation metrics for Natural Language Generation (NLG) shows that task-agnostic metrics have a weak correlation with human . |
| Approach: | They propose a framework to assess the effectiveness of automatic metrics in three NLG tasks . they propose task-agnostic and human-aligned metrics to be used for evaluation . |
| Outcome: | The proposed framework provides access to the evaluation tools for three NLG tasks. |
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe (2023.acl-long)
Copied to clipboard
Xiang Yue, Huseyin Inan, Xuechen Li, Girish Kumar, Julia McAnallen, Hoda Shajari, Huan Sun, David Levitan, Robert Sim
| Challenge: | Privacy concerns have increased in data-driven products due to the tendency of machine learning models to memorize sensitive training data. |
| Approach: | They propose a method for generating useful synthetic text with a formal privacy guarantee by fine-tuning a pretrained generative language model with DP. |
| Outcome: | The proposed method produces synthetic text competitive in terms of utility with its non-private counterpart, while providing strong protection against potential privacy leakages. |
The EuroPat Corpus: A Parallel Corpus of European Patent Data (2022.lrec-1)
Copied to clipboard
| Challenge: | a new corpus of patent-specific parallel data is available for 6 official European languages paired with English: German, Spanish, French, Croatian, Norwegian, and Polish. |
| Approach: | They present a patent-specific corpus of parallel data for 6 official European languages paired with English: German, Spanish, French, Croatian, Norwegian, and Polish. |
| Outcome: | The filtered corpus ranges in size from 51 million sentences (Spanish-English) to 154k sentences (Croatian-English), with the unfiltered (raw) corpus being up to 2 times larger. |
Exploiting Curriculum Learning in Unsupervised Neural Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that the proposed method achieves consistent improvements with faster convergence speed. |
| Approach: | They propose a curriculum learning method to gradually utilize pseudo bi-texts based on their quality from multiple granularities. |
| Outcome: | The proposed method achieves consistent improvements with faster convergence speed on WMT 14 En-Fr, WMT14 En-De, and LDC En-Zh translation tasks. |
Demonstrating ViviDoc: Generating Interactive Documents through Human-Agent Collaboration (2026.acl-demo)
Copied to clipboard
| Challenge: | Recent LLM-based agents can automate content creation, but naively applying them yields uncontrollable and unverifiable outputs. |
| Approach: | They propose a human-agent collaborative system that generates interactive educational documents from a single topic input. |
| Outcome: | The proposed system generates documents comparable in quality to human-authored ones. |
Uncovering Limitations of Large Language Models in Information Seeking from Tables (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for Table Information Seeking (TabIS) are lacking in reliable evaluation. |
| Approach: | They propose a benchmark to evaluate the table information seeking abilities of large language models . they use a single-choice question format instead of a text-based evaluation . |
| Outcome: | The proposed benchmark is more reliable than existing models and is available online. |
DISAPERE: A Dataset for Discourse Structure in Peer Review Discussions (2022.naacl-main)
Copied to clipboard
Neha Kennard, Tim O’Gorman, Rajarshi Das, Akshay Sharma, Chhandak Bagchi, Matthew Clinton, Pranay Kumar Yelugam, Hamed Zamani, Andrew McCallum
| Challenge: | Prior work on labeling arguments extracted from peer review text has focused qualified labor force on labelling arguments extracted by the text. |
| Approach: | They synthesize label sets from prior work and extend them to include fine-grained annotations of review and rebuttal sentences. |
| Outcome: | The proposed dataset synthesizes label sets from prior work and extends them to include fine-grained annotation of review and rebuttal sentences. |
Knowledge-augmented Financial Market Analysis and Report Generation (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods to generate financial market analysis text require extensive financial knowledge and skill of financial analysts. |
| Approach: | They propose a task to generate financial market analysis reports using financial market data using a financial knowledge graph. |
| Outcome: | The proposed framework outperforms large-scale language models and retrieval-augmented baselines in the financial market analysis generation task. |
Transcending Scaling Laws with 0.1% Extra Compute (2023.emnlp-main)
Copied to clipboard
Yi Tay, Jason Wei, Hyung Chung, Vinh Tran, David So, Siamak Shakeri, Xavier Garcia, Steven Zheng, Jinfeng Rao, Aakanksha Chowdhery, Denny Zhou, Donald Metzler, Slav Petrov, Neil Houlsby, Quoc Le, Mostafa Dehghani
| Challenge: | Existing scaling of language models is expensive and requires significant computational costs. |
| Approach: | They propose a method that substantially improves existing language models and their scaling curves with a relatively tiny amount of extra compute. |
| Outcome: | The proposed method significantly improves existing language models and their scaling curves with a relatively tiny amount of extra compute. |
Neural Search Space in Gboard Decoder (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Gboard decoder uses context, a lexicon and language models to provide a user-friendly keyboard. |
| Approach: | They propose a Neural Search Space which replaces an N-gram LM with a neural network LM and dynamically constructs the search space during decoding. |
| Outcome: | The proposed system improves the quality of the decoded keyboards on various locales with acceptable latency increases. |
Semantic Consistency-Based Uncertainty Quantification for Factuality in Radiology Report Generation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Radiology report generation has shown great potential in assisting radiologists . generative medical Vision Large Language Models (VLLMs) are prone to hallucinations and can produce inaccurate diagnostic information. |
| Approach: | They propose a framework that provides both report-level and sentence-level uncertainties. |
| Outcome: | The proposed method improves factuality scores by 10% by rejecting 20% of reports on the MIMIC-CXR dataset. |
Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have great potential for synthetic data generation. |
| Approach: | They show that large language models can generate useful data even for complex tasks . they use a symmetric task difficulty asymmetry to prompt an LLM to generate plausible input text for a target output structure. |
| Outcome: | The proposed approach outperforms existing models by a substantial margin on closed information extraction tasks with 1.8M data points and 770M parameters. |
In-the-wild Audio Spatialization with Flexible Text-guided Localization (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for mapping monaural audio to binaural signals lack flexibility and interactive control needed in complex multi-object user-interactive environments. |
| Approach: | They propose a text-guided audio spatialization framework that utilizes diverse text prompts to evaluate binaural audio models. |
| Outcome: | The proposed framework learns binaural differences guided by 3D spatial location and relative position prompts, enhanced with flipped-channel audio. |
Generative Text Modeling through Short Run Inference (2021.eacl-main)
Copied to clipboard
| Challenge: | Latent variable models for text capture global semantic and syntactic features when trained correctly. |
| Approach: | They propose a short run dynamics for inference that initializes from the prior distribution of the latent variable and runs a small number of Langevin dynamics steps guided by its posterior distribution. |
| Outcome: | The proposed model is able to generate coherent sentences with smooth transition and shows no sign of posterior collapse. |
Is Anisotropy Truly Harmful? A Case Study on Text Clustering (2023.acl-short)
Copied to clipboard
| Challenge: | Contextualized pre-trained representations are widely used as input to various tasks such as information retrieval, anomaly detection and document clustering. |
| Approach: | They propose to examine the impact of different transformations on isotropy and performance to assess the true impact of anisotropi. |
| Outcome: | The proposed model is based on a clustering task and shows that it has limited impact on expressiveness and closeness. |
A Comprehensive Survey of Sentence Representations: From the BERT Epoch to the CHATGPT Era and Beyond (2024.eacl-long)
Copied to clipboard
Abhinav Ramesh Kashyap, Thanh-Tung Nguyen, Viktor Schlegel, Stefan Winkler, See-Kiong Ng, Soujanya Poria
| Challenge: | Sentence representations are a critical component in NLP applications such as retrieval, question answering, and text classification. |
| Approach: | They present a systematic review of the literature on sentence representations focusing mostly on deep learning models. |
| Outcome: | The proposed methods highlight the key contributions and challenges in this area and suggest potential avenues for improving the quality and efficiency of sentence representations. |
CAMEL-Bench: A Comprehensive Arabic LMM Benchmark (2025.findings-naacl)
Copied to clipboard
Sara Ghaboura, Ahmed Heakl, Omkar Thawakar, Ali Husain Salem Abdulla Alharthi, Ines Riahi, Abduljalil Radman, Jorma Laaksonen, Fahad Shahbaz Khan, Salman Khan, Rao Muhammad Anwer
| Challenge: | Recent years have witnessed a significant interest in developing large multimodal models capable of performing various visual reasoning and understanding tasks. |
| Approach: | They propose to use Arabic as a language to evaluate large multi-modal models capable of performing visual reasoning and understanding tasks. |
| Outcome: | The proposed benchmark comprises eight diverse domains and 38 sub-domains to represent a large population of over 400 million speakers. |
A Career Interview Dialogue System using Large Language Model-based Dynamic Slot Generation (2025.coling-main)
Copied to clipboard
Ekai Hashimoto, Mikio Nakano, Takayoshi Sakurai, Shun Shiramatsu, Toshitake Komazaki, Shiho Tsuchiya
| Challenge: | a slot-filling-based interview dialogue system is limited in the flexibility of information collection . authors propose a method that leverages large language models to generate new slots according to the flow of the dialogue . |
| Approach: | They propose a slot-filling dialogue system that collects information on staff careers . they incorporate abduction into the slot generation process to enable more natural conversations . |
| Outcome: | The proposed method improves the efficiency and quality of career interviews conducted by nursing managers. |
DialogCC: An Automated Pipeline for Creating High-Quality Multi-Modal Dialogue Dataset (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing multi-modal dialogue datasets that focus on image-based dialogues have low quality and limited diversity of images per dialogue. |
| Approach: | They propose to construct a multi-modal dialogue dataset that guarantees both dialogue quality and image diversity without requiring minimum human effort. |
| Outcome: | The proposed dataset outperforms existing datasets in terms of quality and diversity in human evaluation. |
CodeHacker: Automated Test Case Generation for Detecting Vulnerabilities in Competitive Programming Solutions (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for Large Language Models often lack coverage for subtle corner cases . a substantial amount of effort has been applied to address this challenge . |
| Approach: | They propose a framework that generates adversarial test cases that expose latent vulnerabilities in code submissions. |
| Outcome: | The proposed framework improves the True Negative Rate (TNR) of existing datasets and generates superior adversarial cases on liveCodeBench. |
WaveFM: A High-Fidelity and Efficient Vocoder Based on Flow Matching (2025.naacl-long)
Copied to clipboard
| Challenge: | Flow matching is a robust and stable approach to training diffusion models, but it can result in subpar audio quality. |
| Approach: | They propose a reparameterized flow matching model for mel-spectrogram conditioned speech synthesis that uses a mel prior instead of a standard Gaussian prior to minimize unnecessary transportation costs. |
| Outcome: | The proposed model improves sample quality and generation speed for speech vocoders while reducing transportation costs. |
SLAM-Omni: Timbre-Controllable Voice Interaction System with Single-Stage Training (2025.findings-acl)
Copied to clipboard
Wenxi Chen, Ziyang Ma, Ruiqi Yan, Yuzhe Liang, Xiquan Li, Ruiyang Xu, Zhikang Niu, Yanqiao Zhu, Yifan Yang, Zhanxun Liu, Kai Yu, Yuxuan Hu, Jinyu Li, Yan Lu, Shujie Liu, Xie Chen
| Challenge: | a new spoken dialogue system with single-stage training is demonstrating its low latency and high quality . SLAM-Omni achieves zero-shot timbre control by modeling spoken language with semantic tokens . |
| Approach: | They propose a timbre-controllable, end-to-end voice interaction system with single-stage training. |
| Outcome: | The proposed system outperforms previous models on 4 GPUs with limited data. |
Addressing Semantic Drift in Generative Question Answering with Auxiliary Extraction (2021.acl-short)
Copied to clipboard
| Challenge: | Recent work focuses on question answering based on machine reading comprehension . current approaches treat QA as extracting a consecutive piece of text to a given question. |
| Approach: | They propose a generative QA model that incorporates an extractive mechanism into a model. |
| Outcome: | The proposed model improves quality and semantic accuracy over baseline models. |
Quality Beyond A Glance: Revealing Large Quality Differences Between Web-Crawled Parallel Corpora (2025.coling-main)
Copied to clipboard
| Challenge: | Parallel corpora play a vital role in advanced multilingual natural language processing tasks, notably in machine translation (MT). |
| Approach: | They manually and automatically evaluated four well-known publicly available parallel corpora across eleven language pairs. |
| Outcome: | The results show that the four well-known parallel corpora have a substantial amount of noisy sentence pairs, while CCMatrix and CCAligned have low quality sentences. |
DialCrowd 2.0: A Quality-Focused Dialog System Crowdsourcing Toolkit (2022.lrec-1)
Copied to clipboard
| Challenge: | DialCrowd 2.0 helps requesters obtain higher quality data from human intelligence tasks. |
| Approach: | They propose to use DialCrowd 2.0 to help requesters obtain higher quality data . they aim to improve the way requesters present tasks and facilitate effective communication with workers. |
| Outcome: | The proposed toolkit enables requesters to obtain higher quality data by presenting tasks more clearly and facilitating effective communication with workers. |
The Devil is in the Details: On the Pitfalls of Vocabulary Selection in Neural Machine Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Neural Machine Translation models can be optimized to improve latency by constraining the set of output words . lexical shortlisting fails to select the right set of input words for semantically non-compositional phenomena such as idiomatic expressions. |
| Approach: | They propose a model of vocabulary selection that constrains the set of allowed output words . they propose to increase the size of the allowed set to restore translation quality . |
| Outcome: | The proposed model restores translation quality of an unconstrained system, as measured by human evaluations on WMT newstest2020 and idiomatic expressions, at an inference latency competitive with alignment-based selection using aggressive thresholds. |
Sentence Bottleneck Autoencoders from Transformer Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for pretraining a language model on text have been used for building models in NLP, but they do not work for sentence representations derived from pretrainer models based on tokens or basic pooling operations. |
| Approach: | They propose to build a sentence-level autoencoder from a pretrained transformer language model. |
| Outcome: | The proposed model achieves better quality than previous methods on text similarity and style transfer tasks while using fewer parameters than large pretrained models. |
BiTIIMT: A Bilingual Text-infilling Method for Interactive Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing IMT systems relying on lexical constrained decoding (LCD) are limited in translation efficiency and quality due to LCD. |
| Approach: | They propose a novel interactive neural machine translation system that uses lexical constraints to decode missing words in a manually revised translation. |
| Outcome: | The proposed system performs significantly better and faster than state-of-the-art IMT on three translation tasks. |
With Measured Words: Simple Sentence Selection for Black-Box Optimization of Sentence Compression Algorithms (2021.eacl-main)
Copied to clipboard
| Challenge: | Sentence Compression is the task of generating a shorter, yet grammatical, version of a given sentence, preserving the essence of the original sentence. |
| Approach: | They propose a Black-Box Optimizer for Compression to find the best candidates for compression . they use a black-box compression algorithm to predict how well each sentence could be compressed . |
| Outcome: | The proposed algorithm improves both accuracy and Rouge-F1-score on three datasets. |
Causal Intervention for Abstractive Related Work Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models ignore the inherent causality during related work generation, leading to spurious correlations which downgrade the models’ generation quality and generalizability. |
| Approach: | They propose a Causal Intervention Module for Related Work Generation (CaM) that captures causal relationships in related work generation and implements causal interventions to mitigate the negative impact of spurious correlations. |
| Outcome: | The proposed framework improves the quality and coherence of generated related work by capturing causalities in the generation process. |
Attack Prompt Generation for Red Teaming and Defending Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies construct attack prompts via manual or automatic methods, but these methods have limitations on cost and quality. |
| Approach: | They propose an attack framework to instruct LLMs to mimic human-generated prompts through in-context learning and a defense framework that fine-tunes victim LLM's through iterative interactions with the attack framework. |
| Outcome: | The proposed approach is based on experiments on different LLMs to evaluate their effectiveness against red teaming attacks. |
Advancing Precise Outline-Conditioned Text Generation with Task Duality and Explicit Outline Control (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing studies on outline-conditioned text generation focus on generating text using provided outlines as rough sketches, but lack of clarity and rationality of the rough outlines hampers quality of the generated text. |
| Approach: | They propose a novel task that requires generating stories based on specific, sentence-level outlines. |
| Outcome: | The proposed framework improves the quality of precise outline-conditioned text generation. |
AttributeForge: An Agentic LLM Framework for Automated Product Schema Modeling (2025.emnlp-industry)
Copied to clipboard
Yunhan Huang, Klevis Ramo, Andrea Iovine, Melvin Monteiro, Sedat Gokalp, Arjun Bakshi, Hasan Turalic, Arsh Kumar, Jona Neumeier, Ripley Yates, Rejaul Monir, Simon Hartmann, Tushar Manglik, Mohamed Yakout
| Challenge: | e-commerce platforms are producing only tens of attributes per month for schema modeling . authors present a framework to automate end-to-end product schema modeling using Large Language Models . |
| Approach: | They introduce a framework to automate end-to-end product schema modeling using Large Language Models. |
| Outcome: | The proposed framework achieves an 88 increase in modeling throughput while delivering superior quality. |
Predicting Annotation Difficulty to Improve Task Routing and Model Performance for Biomedical Information Extraction (N19-1)
Copied to clipboard
| Challenge: | Modern NLP systems require high-quality annotations, but experts are expensive and lay annotators may not have the knowledge to provide high- quality annotations. |
| Approach: | They propose to directly model instance difficulty to improve model performance and to route instances to appropriate annotators. |
| Outcome: | The proposed model improves performance on a biomedical information extraction task using expert and lay annotations. |
Estimating Summary Quality with Pairwise Preferences (N18-1)
Copied to clipboard
| Challenge: | Existing evaluation systems rely on gold standard summaries but they are expensive and require the availability of experts to achieve high quality. |
| Approach: | They propose an alternative evaluation approach based on pairwise preferences of sentences to provide useful feedback in the form of pairwise preference. |
| Outcome: | The proposed evaluation framework performs better than the three most popular versions of ROUGE with less expensive human input. |
Leveraging Explicit Reasoning for Inference Integration in Commonsense-Augmented Dialogue Models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to commonsense-augmented dialogue rely on implicit reasoning to integrate commonsensense inferences during response generation. |
| Approach: | They propose to separate commonsense reasoning into explicit steps for generating, selecting, and integrating commonsensense into dialogue responses. |
| Outcome: | The proposed model infers commonsense knowledge from dialogue contexts to improve response quality and naturalness of dialogue interactions. |
A Neural Approach to Pun Generation (P18-1)
Copied to clipboard
| Challenge: | generating puns with artificial intelligence techniques requires manual training and templates. |
| Approach: | They propose neural network models for homographic pun generation that can generate puns without requiring any pun data for training. |
| Outcome: | The proposed models generate homographic puns of good readability and quality without training. |
TriSum: Learning Summarization Ability from Large Language Models with Structured Rationale (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have advanced tasks like text summarization, but their size and computational demands limit their use in resource-constrained and privacy-centric settings. |
| Approach: | They propose a framework for distilling LLMs’ text summarization abilities into a compact, local model using a curriculum learning strategy that evolves from simple to complex tasks. |
| Outcome: | The proposed framework outperforms baseline models on CNN/DailyMail, XSum, and ClinicalTrial, and improves interpretability by providing insights into the summarization rationale. |
Alignment verification to improve NMT translation towards highly inflectional languages with limited resources (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to improve translation quality using limited training data are phrase-based and syntax-based approaches. |
| Approach: | They propose to combine a neural MT system with an open source module to improve translation quality. |
| Outcome: | The proposed method improves translation quality over the best individual NMT and the standard ensemble system provided in the Marian-NMT system. |
CoCoa: An Encoder-Decoder Model for Controllable Code-switched Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Generating code-switched text with fine-grained control on the degree of code-witching and the lexical choices used to convey formality has been well-explored. |
| Approach: | They propose to generate code-switched text with fine-grained control on the degree of code-changing and lexical choices used to convey formality. |
| Outcome: | The proposed model can be invoked at test-time to synthesize code-switched text faithful to syntactic and lexical attributes relevant to code-witching. |
On Systematic Style Differences between Unsupervised and Supervised MT and an Application for High-Resource Machine Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Modern unsupervised machine translation systems reach reasonable translation quality under clean and controlled data conditions. |
| Approach: | They compare unsupervised and supervised machine translation systems of similar quality . they combine the benefits of both methods into a single system . |
| Outcome: | The proposed system improves adequacy and fluency as measured by human evaluators. |
Do Neural Topic Models Really Need Dropout? Analysis of the Effect of Dropout in Topic Modeling (2023.eacl-main)
Copied to clipboard
| Challenge: | Dropout is a regularization trick used to resolve overfitting in large feedforward neural networks, but there is nil analysis of it for unsupervised models and in particular, VAE-based neural topic models. |
| Approach: | They propose to use dropout to solve overfitting problems in unsupervised neural topic models by stochastically dropping out the activation of neurons to prevent complex co-adaptations of feature vectors. |
| Outcome: | The proposed class of neural topic models can be used to improve the quality and predictive performance of the generated topics. |
Synthetic Data Made to Order: The Case of Parsing (D18-1)
Copied to clipboard
| Challenge: | supervised dependency parsing is a core task in natural language processing, but unsupervised parsers can hardly produce useful parses. |
| Approach: | They propose to permute the constituents of an existing dependency treebank so that its surface part-of-speech statistics approximately match those of the target language. |
| Outcome: | The proposed method improves the parsing accuracy of a target language . the proposed method is based on a distribution of gold POS bigrams . |
Attention Focusing for Neural Machine Translation by Bridging Source and Target Embeddings (P18-1)
Copied to clipboard
| Challenge: | Neural machine translation uses source and target word embeddings to improve translation quality . source and targeted word embeds are at the two ends of a long information processing procedure . |
| Approach: | They propose a method to shorten the distance between source and target words in neural machine translation by bridging source and targeting word embeddings. |
| Outcome: | The proposed method shortens the distance between source and target words in neural machine translation and strengthens their association. |
End-to-End Construction of NLP Knowledge Graph (2021.findings-acl)
Copied to clipboard
| Challenge: | a new schema for NLP knowledge about tasks, datasets and metrics is proposed. |
| Approach: | They propose a new schema that represents knowledge about tasks, datasets and metrics in the NLP domain. |
| Outcome: | The proposed framework can be automatically built into scientific leaderboards . the proposed system achieves reasonable results for all relation types on this small-scale graph . |
Unifying Human and Statistical Evaluation for Natural Language Generation (N19-1)
Copied to clipboard
| Challenge: | Human evaluation captures quality but fails to capture diversity . statistical evaluation fails to catch models that plagiarize from training set . |
| Approach: | They propose a framework which evaluates both diversity and quality based on the optimal error rate of predicting whether a sentence is human-generated. |
| Outcome: | The proposed framework evaluates diversity and quality on summarization and chit-chat dialogue. |
Accuracy is not enough: Evaluating Personalization in Summarizers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing accuracy measures cannot evaluate the degree of personalization of summarization models. |
| Approach: | They propose to use a PENS dataset to analyze the degree of personalization of ten different summarization models. |
| Outcome: | The proposed measure can evaluate the degree of personalization of summarization models using the PENS dataset. |
SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for relation extraction only implicitly learn to model relevant contexts and entity types while being trained for RE. |
| Approach: | They propose to explicitly teach the model to capture relevant contexts and entity types by supervising and augmenting intermediate steps (SAIS) for RE. |
| Outcome: | The proposed method outperforms the runner-up method on three benchmarks by 5.04% . textual contexts and entity types are the major information sources that lead to the success of previous approaches. |
Syntax-guided Localized Self-attention by Constituency Syntactic Distance (2022.findings-emnlp)
Copied to clipboard
Shengyuan Hou, Jushi Kai, Haotian Xue, Bingyu Zhu, Bo Yuan, Longtao Huang, Xinbing Wang, Zhouhan Lin
| Challenge: | Recent studies have shown that Transformers is implicitly learning syntactic information from data, albeit is highly dependent on the quality and scale of the training data. |
| Approach: | They propose a syntax-guided localized self-attention model that allows directly incorporating grammar structures from an external constituency parser. |
| Outcome: | The proposed model improves translation performance on a variety of datasets, from small to large datasets and with different source languages. |
Mark-Evaluate: Assessing Language Generation using Population Estimation Methods (2020.coling-main)
Copied to clipboard
| Challenge: | Existing population estimation methods focus on open populations or closed populations, but our methods show a higher correlation to human evaluation than existing metrics on several challenging tasks. |
| Approach: | They propose a family of metrics to assess language generation derived from population estimation methods widely used in ecology. |
| Outcome: | The proposed methods show a higher correlation to human evaluation than existing metrics on several challenging tasks, namely unconditional language generation, machine translation, and text summarization. |
mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus (2025.findings-acl)
Copied to clipboard
Matthieu Futeral, Armel Randy Zebaze, Pedro Ortiz Suarez, Julien Abadji, Rémi Lacroix, Cordelia Schmid, Rachel Bawden, Benoît Sagot
| Challenge: | Existing studies show that multimodal large language models can learn from text-image data. |
| Approach: | They propose to train multimodal large language models on large amounts of text-image data . they also show a boost in few-shot learning performance across various multilingual tasks . |
| Outcome: | The proposed dataset is not public and is only in English . it is the first large-scale multilingual and multimodal document corpus crawled from the web. |
JESC: Japanese-English Subtitle Corpus (L18-1)
Copied to clipboard
| Challenge: | Existing data on Japanese-English subtitles are limited due to the high cost of manual construction. |
| Approach: | They describe the Japanese-English Subtitle Corpus by crawling and aligning subtitles found on the web. |
| Outcome: | The JESC dataset covers the underrepresented domain of conversational dialogue. |
LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks (2025.acl-long)
Copied to clipboard
Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li
| Challenge: | Existing benchmarks on longcontext large language models fail to reflect their deep understanding capabilities across diverse tasks. |
| Approach: | They propose a benchmark to assess the ability of long-context large language models to handle long-text problems. |
| Outcome: | The proposed model achieves 50.1% accuracy when directly answering the questions . human experts achieve only 53.7% accuracy under a 15-minute time constraint . |
Towards Better Evaluation for Generated Patent Claims (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies highlight inconsistencies between automated evaluation metrics and human expert assessments for patent claims. |
| Approach: | They propose a multi-dimensional evaluation method specifically designed for patent claims that incorporates features annotated by patent experts. |
| Outcome: | The proposed method achieves highest correlation with human expert evaluations across all assessment criteria across all tested metrics. |
RAG-Instruct: Boosting LLMs with Diverse Retrieval-Augmented Instructions (2025.emnlp-main)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) has emerged as a key paradigm for enhancing large language models by incorporating external knowledge. |
| Approach: | They propose a method for synthesizing diverse and high-quality RAG instruction data based on any source corpus. |
| Outcome: | The proposed method outperforms existing methods in multiple tasks and achieves strong zero-shot performance. |
High-quality argumentative information in low resources approaches improve counter-narrative generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that fine-tuning improves the performance of language models . large language models generate acceptable texts in a number of scenarios, a study shows . |
| Approach: | They show that fine-tuning improves the task of hate speech counter-narrative generation . they provide a subset of arguments and a good base model is required for the fine-uning to have a positive impact. |
| Outcome: | The proposed model produces counter-narratives that are as satisfactory as the whole set. |
Data Diversity Matters for Robust Instruction Tuning (2024.findings-emnlp)
Copied to clipboard
Alexander Bukharin, Shiyang Li, Zhengyang Wang, Jingfeng Yang, Bing Yin, Xian Li, Chao Zhang, Tuo Zhao, Haoming Jiang
| Challenge: | Recent studies have shown that by curating high quality and diverse instruction tuning datasets, we can significantly improve instruction-following capabilities. |
| Approach: | They propose an algorithm to control diversity and quality of instruction tuning datasets and validate it. |
| Outcome: | The proposed algorithm significantly improves worst and average case performance on large scale instruction tuning datasets. |
Eliciting Affective Events from Language Models by Multiple View Co-prompting (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to generate training data using weakly labeled data are costly and limited . |
| Approach: | They propose a method for acquiring and labeling affective events with multiple view co-prompting using pre-trained language models. |
| Outcome: | The proposed approach improves state-of-the-art affective event classifier on two datasets. |
Beyond Sentence-Level End-to-End Speech Translation: Context Helps (2021.acl-long)
Copied to clipboard
| Challenge: | Document-level contextual information has shown benefits to text-based machine translation, but whether and how it helps end-to-end speech translation is still under-studied. |
| Approach: | They propose a concatenation-based ST model with adaptive feature selection for computational efficiency. |
| Outcome: | The proposed model improves translation quality and robustness to (artificial) audio segmentation errors. |
MECI: A Multilingual Dataset for Event Causality Identification (2022.coling-1)
Copied to clipboard
| Challenge: | Event Causality Identification (ECI) is a task of detecting causal relations between events mentioned in text. |
| Approach: | They propose a multilingual dataset that provides consistent annotations for event causality relations in five languages. |
| Outcome: | The proposed dataset provides consistent annotation guidelines for five languages . the dataset can provide ample research challenges and directions for future research . |
An End-to-End Submodular Framework for Data-Efficient In-Context Learning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in natural language tasks leverage the emergent In-Context Learning ability of pretrained Large Language Models (LLMs). |
| Approach: | They propose a framework for exemplar selection for in-context learning that uses a pool-based active learning approach to select Diverse and informative exemplars from the target tasks’ unlabeled pool. |
| Outcome: | The proposed framework outperforms existing methods for data annotation and similarity-based methods for test query-specific exemplar retrieval on 7 different NLP datasets and 5 LLMs of varying complexities. |
Towards Enhancing Faithfulness for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has achieved great success due to the ability to generate high-quality sentences. |
| Approach: | They propose a training strategy with a multi-task learning paradigm to build a faithfulness enhanced NMT model. |
| Outcome: | The proposed model can generate high-quality sentences that are very close to natural language. |
A Scalable Framework for Automated NER Annotation Correction in Low-Resource Languages (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing NER benchmarks lack quality annotations, resulting in poor performance. |
| Approach: | They propose a frequency-based iterative approach that leverages self-training and a dual-threshold mechanism to enhance inference confidence. |
| Outcome: | The proposed approach improves NER performance on three datasets with a high number of missing annotations. |
Paraphrase Generation and Evaluation on Colloquial-Style Sentences (2020.lrec-1)
Copied to clipboard
| Challenge: | a new study investigates the quality and novelty of generated paraphrases . paraphrase models can be used for information retrieval and data mining . |
| Approach: | They use state-of-the-art neural machine translation models trained on the Opusparcus corpus to generate paraphrases in six languages. |
| Outcome: | The proposed model outperforms the existing model on human evaluation in five of the six languages. |
Non-Repeatable Experiments and Non-Reproducible Results: The Reproducibility Crisis in Human Evaluation in NLP (2023.findings-acl)
Copied to clipboard
| Challenge: | reproducibility of human evaluations is rarely queried in NLP . authors estimate that just 5% of humanevaluations are repeatable . |
| Approach: | They propose to make human evaluations more repeatable and more reproducible . they estimate that just 5% of human evaluation experiments are repeatable . |
| Outcome: | The results show that human evaluations are rarely queried or formally tested in NLP . the authors estimate that just 5% of human evaluation experiments are repeatable . |
Improving Grammatical Error Correction Models with Purpose-Built Adversarial Examples (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for grammatical error correction are data-hungry and it is hard to train a seq2seq model with good performance without suf-Clean. |
| Approach: | They propose a method inspired by adversarial training to generate more meaningful and valuable training examples by continually identifying weak spots of a model and to enhance the model by gradually adding adversarials to the training set. |
| Outcome: | The proposed method improves generalization and robustness of GEC models by adding adversarial examples to the training set. |
Reward Mixology: Crafting Hybrid Signals for Reinforcement Learning Driven In-Context Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for in-context learning (ICL) performance rely on quality and ordering of demonstrations. |
| Approach: | They propose a method that models iterative demonstration selection as a Markov Decision Process and craft hybrid reward signals. |
| Outcome: | The proposed method combines outcome-based accuracy signals with process-oriented signals like stepwise influence and label entropy improvement. |
COMETA: A Corpus for Medical Entity Linking in the Social Media (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets for Entity Linking (EL) fail to address the complex nature of health terminology in layman’s language. |
| Approach: | They propose to use a corpus of 20k English biomedical entity mentions from Reddit expert-annotated with links to a widely-used medical knowledge graph to investigate the ability of these systems to perform complex inference on entities and concepts. |
| Outcome: | The proposed corpus satisfies a combination of desirable properties, from scale and coverage to diversity and quality, that to the best of our knowledge has not been met by existing resources in the field. |
We Are What We Repeatedly Do: Improving Long Context Instruction Following (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language model context lengths have increased by at least 1000 in the past seven years . however, longer contexts pose challenges to system instruction following . |
| Approach: | They propose to formalize verifiable instructions to evaluate model compliance . they implement and evaluate six mitigation strategies to enhance instruction compliance in extended contexts. |
| Outcome: | The proposed model performs better in long contexts than in natural language models. |
Community-Driven Crowdsourcing: Data Collection with Local Developers (L18-1)
Copied to clipboard
| Challenge: | a community-driven approach to annotation applications and crowdsourcing programs is feasible, says a new study. |
| Approach: | They propose to partner with local developers to create custom annotation applications . they recruit and motivate crowd contributors from their communities to perform an annotation task . |
| Outcome: | The proposed approach combines local developers' knowledge of their social networks to collect labeled data. |
AI-Assisted Human Evaluation of Machine Translation (2025.naacl-long)
Copied to clipboard
| Challenge: | Annotation metrics are misaligned with the ideal measure of text quality and human evaluation remains the most accurate, reliable, and ultimate standard. |
| Approach: | They propose an annotation protocol that helps annotators mark erroneous parts of the translation and assign a final score. |
| Outcome: | The proposed protocol reduces the time per span annotation by half . the method reduces annotation budget by 25% with filtering of examples that the AI deems to be likely to be correct. |
An Empirical Study of Speculative Decoding for Small Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing studies focus on 7B-70B parameters models, leaving a knowledge gap for small language models. |
| Approach: | They propose a draft-then-verify paradigm that allows for a single forward pass through a model and transfer of all model parameters to the GPU cache. |
| Outcome: | The proposed method can be used to accelerate small language models with low computational overhead. |
RPO: Retrieval Preference Optimization for Robust Retrieval-Augmented Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models struggle to evaluate the correctness of non-parametric knowledge when it differs from internal memorization, leading to knowledge conflicts during response generation. |
| Approach: | They propose a lightweight alignment method to leverage multi-source knowledge based on retrieval relevance. |
| Outcome: | Experiments on four datasets show that the proposed method outperforms RAG by 4-10% in accuracy without any extra component. |
FinSight: Towards Real-World Financial Deep Research (2026.acl-long)
Copied to clipboard
| Challenge: | FinSight is the first multi-agent framework for automating end-to-end professional, multimodal financial reports. |
| Approach: | They propose a code agent with variable memory architecture that unifies data, tools, and agents into a programmable variable space. |
| Outcome: | The proposed framework outperforms leading deep research systems in factual accuracy, analytical depth, and presentation quality. |
Add-One-In: Incremental Sample Selection for Large Language Models via a Choice-Based Greedy Paradigm (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies focus on individual quality and do not assess the value of training data. |
| Approach: | They propose a choice-based sample selection framework that evaluates sample quality . they use LLMs to evaluate the value of each option during the selection process . |
| Outcome: | The proposed model outperforms the full dataset and recent studies on a larger medical dataset. |
No Context Needed: Contextual Quandary In Idiomatic Reasoning With Pre-Trained Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | idiomatic expressions (IEs) are a non-compositional aspect of a text that makes it difficult for a model to comprehend . general purpose PTLMs are negatively affected by the context, as performance increases with its removal. |
| Approach: | They propose to use idiomatic expressions to infer additional meaning from IEs . they argue that only IE-aware models are suitable for idiom- matic reasoning tasks . |
| Outcome: | The proposed models can reason in the presence of idiomatic expressions, the authors show . they show that general purpose PTLMs are negatively affected by the context . |
Bandits Don’t Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Bandits (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Training data for machine translation (MT) is often sourced from multiple large corpora that are multi-faceted in nature. |
| Approach: | They propose to optimize the balance between translationese and natural training data to relieve system developers from manual schedule design. |
| Outcome: | The proposed model relieves system developers from manual schedule design. |
Finding Replicable Human Evaluations via Stable Ranking Probability (2024.naacl-long)
Copied to clipboard
| Challenge: | a recent study shows that human evaluation is the best way to rank natural language generation systems . human raters can exhibit different behaviors when rating outputs, causing ranking to be unstable . stability is the degree to which a specific evaluation methodology produces the same system ranking when repeated. |
| Approach: | They propose to evaluate results through the lens of stability: stability is the degree to which a specific evaluation methodology produces the same system ranking when repeated. |
| Outcome: | The proposed model is based on a dataset of multi-segment translations rated by multiple professionals . human raters can exhibit different behaviors when rating NLG outputs, the study shows . |
Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing studies on multimodality in simultaneous machine translation have highlighted the challenges for the agent to maintain good translation quality while learning an optimal translation path. |
| Approach: | They propose a multimodal approach to simultaneous machine translation using reinforcement learning with strategies to integrate visual and textual information in both the agent and the environment. |
| Outcome: | The proposed multimodal approach improves translation quality while keeping latency low while providing visual cues. |
Critic-Guided Decoding for Controlled Text Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has demonstrated reinforcement learning and weighted decoding as effective approaches to achieve a higher level of language control and quality with pros and cons. |
| Approach: | They propose a method that combines reinforcement learning and weighted decoding to train a critic from reward models. |
| Outcome: | The proposed method generates more coherent and well-controlled texts than previous methods on three controlled generation tasks, topic control, sentiment control, and detoxification. |
Multilingual Mix: Example Interpolation Improves Multilingual Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to train multilingual models to learn the inductive bias of a shared vocabulary and set of parameters across languages. |
| Approach: | They propose to use a multilingual crossover encoder-decoder to fuse language pairs at an instance level to encourage sharing of input and output spaces. |
| Outcome: | The proposed approach improves quality on English-to-Many, Many-to English and zero-shot translation tasks from +0.5 BLEU up to +5.5 BLUE points. |
LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing (2024.emnlp-main)
Copied to clipboard
Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Cheng Jiayang, Zhaowei Wang, Ying Su, Raj Shah, Ruohao Guo, Jing Gu, Haoran Li, Kangda Wei, Zihao Wang, Lu Cheng, Surangika Ranathunga, Meng Fang, Jie Fu, Fei Liu, Ruihong Huang, Eduardo Blanco, Yixin Cao, Rui Zhang, Philip Yu, Wenpeng Yin
| Challenge: | a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities . |
| Approach: | They present a comparative analysis to identify and distinguish LLM activities from human activities. |
| Outcome: | The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities. |
Translating Translationese: A Two-Step Approach to Unsupervised Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Using a dictionary, given a rough, target language natives can uncover the latent, fully-fluent rendering of the translation. |
| Approach: | They propose a method that breaks translation into two steps by generating a dictionary and then ‘translating’ the resulting pseudo-translation into a fully fluent translation. |
| Outcome: | The proposed method 'gets better translation results on high-resource languages than previously published unsupervised MT studies' |
CogGen: A Cognitively Inspired Recursive Framework for Deep Research Report Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to deep research report generation rely on rigid predefined linear workflows, which cause error accumulation and limit in-depth multimodal fusion and report quality. |
| Approach: | They propose a Cognitively inspired recursive framework for deep research report Generation that simulates cognitive writing and abstract visual representation (AVR) they also propose CLEF, a cognitive load evaluation framework, and a benchmark from our world in data. |
| Outcome: | The proposed framework achieves state-of-the-art among open-source systems, surpassing Gemini Deep Research. |
ResoFilter: Fine-grained Synthetic Data Filtering for Large Language Models through Data-Parameter Resonance Analysis (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for enhancing large language models lack clear metrics for evaluating data characteristics. |
| Approach: | They propose a method that integrates models, data, and tasks to refine datasets. |
| Outcome: | The proposed method achieves comparable results to full-scale fine-tuning using only half the data in mathematical tasks and exhibits strong generalization across different models and domains. |
A Prospective-Performance Network to Alleviate Myopia in Beam Search for Response Generation (C18-1)
Copied to clipboard
| Challenge: | Existing methods to generate responses using beam search focus on current optimal results. |
| Approach: | They propose a beam search method that uses a Prospective-Performance Network to predict the future reward of a partially-generated response. |
| Outcome: | The proposed method can increase the quality and diversity of generated responses with high inference efficiency. |
Explicit Query Rewriting for Conversational Dense Retrieval (2022.emnlp-main)
Copied to clipboard
| Challenge: | In a conversational search scenario, a query might be context-dependent because some words are referred to previous expressions or omitted. |
| Approach: | They propose a model that performs query rewriting and context modelling in a unified framework by highlighting relevant terms in the query context. |
| Outcome: | The proposed model outperforms baseline models in terms of quality of query rewriting and quality of contextualized query embedding. |
Se2: Sequential Example Selection for In-Context Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Prior work has explored the selection of examples for in-context learning, neglecting the internal relationships between examples and exist an inconsistency between training and inference. |
| Approach: | They propose a sequential-aware method that leverages the LLM’s feedback on varying context, aiding in capturing inter-relationships and sequential information among examples. |
| Outcome: | Experiments on 23 NLP tasks show that Se2 surpasses baselines and achieves 42% relative improvement over random selection. |
Quality and Efficiency of Manual Annotation: Pre-annotation Bias (2022.lrec-1)
Copied to clipboard
| Challenge: | Annotators using pre-annotation are less efficient at producing high quality annotations. |
| Approach: | They propose to use an automatic pre-annotation for a task to judge annotation quality . they also evaluate the effect of automatic linguistically-based checks on the same data . |
| Outcome: | The proposed method improves the quality of annotated sentences without reducing quality. |
AutoMedEval: Harnessing Language Models for Automatic Medical Capability Evaluation (2025.acl-long)
Copied to clipboard
Xiechi Zhang, Zetian Ouyang, Linlin Wang, Gerard De Melo, Zhu Cao, Xiaoling Wang, Ya Zhang, Yanfeng Wang, Liang He
| Challenge: | Existing evaluation methods based on large language models (LLMs) are expensive and lack expertise due to limitations in human expertise. |
| Approach: | They propose an open-source automatic evaluation model with 13B parameters specifically engineered to measure the question-answering proficiency of medical LLMs. |
| Outcome: | The proposed model surpasses baselines in terms of correlation with human judgments. |
FOAM: A Follower-aware Speaker Model For Vision-and-Language Navigation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing speaker-follower models are follower-agnostic and fail to take state of follower into account. |
| Approach: | They propose a speaker-follower model that is constantly updated given follower feedback . they optimize the speaker and obtain its training signals by evaluating the follower on labeled data . |
| Outcome: | The proposed model outperforms strong baseline models on room-to-room and room-across-room datasets. |
Medical Text Simplification: Optimizing for Readability with Unlikelihood Training and Reranked Beam Search Decoding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Text simplification has emerged as an increasingly useful application of AI for bridging the communication gap in specialized fields such as medicine, where the lexicon is often dominated by technical jargon and complex constructs. |
| Approach: | They propose a unlikelihood loss that encourages generation of simpler terms and a reranked beam search decoding method that optimizes for simplicity. |
| Outcome: | The proposed methods achieve better performance on readability metrics on three datasets. |
TANDO: A Corpus for Document-level Machine Translation (2022.lrec-1)
Copied to clipboard
Harritxu Gete, Thierry Etchegoyhen, David Ponce, Gorka Labaka, Nora Aranberri, Ander Corral, Xabier Saralegi, Igor Ellakuria, Maite Martin
| Challenge: | Document-level Neural Machine Translation aims to increase the quality of neural translation models by taking into account contextual information. |
| Approach: | They propose to use document-level corpus for Basque-Spanish language pairs to take into account contextual information and perform fine-grained evaluations of gender and gender. |
| Outcome: | The proposed corpus is suitable for fine-grained evaluation of document-level machine translation systems. |
Can Synthetic Translations Improve Bitext Quality? (2022.acl-long)
Copied to clipboard
| Challenge: | Synthetic translations have been used for a wide range of NLP tasks, but it remains unclear how they differ from naturally occurring data. |
| Approach: | They propose to use a semantic equivalence classifier to improve bitext quality without additional bilingual supervision to replace the originals. |
| Outcome: | The proposed samples improve bitext quality without additional bilingual supervision and are validated intrinsically and extrinsically through bilingual induction and MT tasks. |
Tree-Structured Non-Autoregressive Decoding for Sequence-to-Sequence Text Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Autoregressive Transformers suffer from high inference latency due to sequential token generation. |
| Approach: | They propose a tree-structured non-autoregressive decoding paradigm that bridges autoregressive and non-automatic decoding. |
| Outcome: | The proposed paradigm outperforms autoregressive and non-autoregressive decoding in machine translation and paraphrase generation. |
Decoding Speculative Decoding (2025.naacl-long)
Copied to clipboard
| Challenge: | Speculative decoding is a widely used technique to speed up inference for Large Language Models (LLMs) Autoregressive decoding has been known to be hardware inefficient, leading to poor resource utilization and low throughput during inference. |
| Approach: | They propose to use a draft model to generate speculative tokens and then use the target LLM to verify those tokens. |
| Outcome: | The proposed model can provide 111% higher throughput than existing draft models and generalizes further to all LLaMA models and supervised fine-tuned models. |
Evaluating Machine Translation Datasets for Low-Web Data Languages: A Gendered Lens (2026.findings-acl)
Copied to clipboard
Hellina Hailu Nigatu, Bethelhem Yemane Mamo, Bontu Fufa Balcha, Debora Taye Tesfaye, Elbethel Daniel Zewdie, Ikram Behiru Nesiru, Jitu Ewnetu Hailu, Senait Mengesha Yayo
| Challenge: | afan oromo, amharic, and tigrinya are low-resourced languages . they are used for training, benchmarks, news, health, and sports . afono o'mara: quantity does not guarantee quality of MT datasets . |
| Approach: | They investigate the quality of machine translation datasets for three low-resourced languages . they found a large skew towards the male gender in the datasets . |
| Outcome: | The results show that training data has large representation of political and religious text, but benchmark datasets focus on news, health, and sports. |
Multi-Hall-SA: A Cross-lingual Benchmark for Multi-Type Hallucination Detection in Low-Resource South African Languages (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models generate false or unsupported information, which can be difficult to detect in low-resource languages. |
| Approach: | They propose a cross-lingual benchmark for hallucination detection spanning English and South African languages. |
| Outcome: | The proposed model detects 23.6% fewer hallucinations in South African languages compared to English . human validation confirms the quality and cross-lingual alignment of the model . |
Massive vs. Curated Embeddings for Low-Resourced Languages: the Case of Yorùbá and Twi (2020.lrec-1)
Copied to clipboard
| Challenge: | a recent study shows that word embeddings can be useful for training downstream natural language processing tasks. |
| Approach: | They compare word embeddings obtained by word embeds from curated corpora with a language-dependent processing. |
| Outcome: | The proposed model compares word embeddings with word embeds from curated corpora and a language-dependent processing on two African languages. |
Measuring Robustness for NLP (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to evaluate NLP models are limited to news domains and cannot be generalized to other domains. |
| Approach: | They propose a measure of NLP quality based on robustness . they measure consistency of cross-domain accuracy and introduce coefficient of variation and gamma-Robustness based upon human evaluation . |
| Outcome: | The proposed approach shows higher agreement with human evaluation than accuracy scores on ranking machine translation systems. |
Tracking Life’s Ups and Downs: Mining Life Events from Social Media Posts for Mental Health Analysis (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies have indicated that major life events can greatly impact individuals’ mental health, but shedding its light on social media data is challenging due to the complexity and ambiguity nature of life events. |
| Approach: | They propose to extract life events mentioned in posts on social media to uncover a social media event dataset which includes 12 major life event categories that are likely to occur in everyday life. |
| Outcome: | The proposed dataset includes 12 life event categories that are likely to occur in everyday life and is human-annotated under iterative procedure and boasts a high level of quality. |
Vietnamese Automatic Speech Recognition: A Revisit (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing datasets with low quality and inconsistent annotations are insufficient for high-quality models. |
| Approach: | They propose a pipeline for aggregating and preprocessing high-quality ASR datasets from diverse, potentially noisy, open-source sources. |
| Outcome: | The proposed pipeline provides a foundation for training and evaluating state-of-the-art Vietnamese ASR systems. |
MIT-10M: A Large Scale Parallel Corpus of Multilingual Image Translation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing datasets suffer from limitations in scale, diversity, and quality, hindering the development and evaluation of IT models. |
| Approach: | They propose a large-scale parallel corpus of multilingual image translation with over 10M image-text pairs derived from real-world data. |
| Outcome: | The proposed model performs better in tackling challenging and complex image translation tasks in the real world. |
EPiDA: An Easy Plug-in Data Augmentation Framework for High Performance Text Classification (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for data augmentation do not fully exploit the potential of DA in NLP. |
| Approach: | They propose an easy and plug-in framework for data augmentation to support effective text classification. |
| Outcome: | The proposed framework outperforms existing methods in most cases, but not using agent networks or pre-trained generation networks. |
Revisiting Pivot-Based Paraphrase Generation: Language Is Not the Only Optional Pivot (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for paraphrase generation rely on language as the pivot . however, there is no evidence that parallel data of paraphrases is needed for paraphrasing. |
| Approach: | They propose to use semantic and syntactic representations as pivot for paraphrase generation. |
| Outcome: | The proposed method can generate paraphrases with better quality than using language as pivot. |
OpenForecast: A Large-Scale Open-Ended Event Forecasting Dataset (2025.coling-main)
Copied to clipboard
| Challenge: | Existing closed-ended event forecasting methods are constrained by a limited answer space. |
| Approach: | They introduce OpenForecast, a large-scale open-ended dataset with three open-ending event forecasting tasks and an automatic LLM-based method for complex events. |
| Outcome: | The proposed method can be used to evaluate the ability of complex event forecasting of large language models. |
When to Trust LLMs: Aligning Confidence with Response Quality (2024.findings-acl)
Copied to clipboard
Shuchang Tao, Liuyi Yao, Hanxing Ding, Yuexiang Xie, Qi Cao, Fei Sun, Jinyang Gao, Huawei Shen, Bolin Ding
| Challenge: | Existing methods express reliability by confidence level, but lack objective guidance . Existing approaches express reliability but lack guidance on when to trust LLMs . |
| Approach: | They propose a reward-based approach to align confidence with quality to ensure reliability . they propose 'conqORD' to help model to verbalize greater confidence for higher quality responses . |
| Outcome: | Experiments show that CONQORD significantly improves confidence and response accuracy . the proposed approach can be used to determine reliability of large language models . |
Norm-based Noisy Corpora Filtering and Refurbishing in Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing noisy corpora filtering methods are insufficient to solve this problem, requiring multiple scorers trained on clean bitexts. |
| Approach: | They propose to use the information ratio from the source to the target side to distinguish unparallel sentence pairs by using norms of context vectors. |
| Outcome: | The proposed method performs comparably with state-of-the-art noisy corpora filtering techniques but is more efficient and easier to operate. |
TeleMelody: Lyric-to-Melody Generation with a Template-Based Two-Stage Method (2022.emnlp-main)
Copied to clipboard
Zeqian Ju, Peiling Lu, Xu Tan, Rui Wang, Chen Zhang, Songruoyao Wu, Kejun Zhang, Xiang-Yang Li, Tao Qin, Tie-Yan Liu
| Challenge: | a new lyric-to-melody generation system bridges the gap between lyrics and melodies . previous generation systems lack paired data and lack of control on generated melodie. |
| Approach: | They develop a lyric-to-melody generation system with music template to bridge the gap between lyrics and melodies. |
| Outcome: | The proposed system bridges the gap between lyrics and melodies by using music template. |
Employing Argumentation Knowledge Graphs for Neural Argument Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generating arguments use end-to-end knowledge graphs or are controlled with respect to the argument's topic, aspects, or stance. |
| Approach: | They construct and populate three knowledge graphs and encode them into debate portals and relevant paragraphs from Wikipedia. |
| Outcome: | The proposed model produces arguments with superior quality than those generated without knowledge. |
Surprisal and Metaphor Novelty Judgments: Moderate Correlations and Divergent Scaling Effects Revealed by Corpus-Based and Synthetic Datasets (2026.eacl-long)
Copied to clipboard
| Challenge: | Novel metaphor comprehension involves complex semantic processes and linguistic creativity. |
| Approach: | They propose a cloze-style surprisal method that conditions on full-sentence context. |
| Outcome: | The proposed method shows that LM surprisal yields moderate correlations with scores/labels of metaphor novelty. |
SQLWOZ: A Realistic Task-Oriented Dialogue Dataset with SQL-Based Dialogue State Representation for Complex User Requirements (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing TOD datasets present simplified interactions with simple slot-value style constraints and preferences. |
| Approach: | They propose a novel TOD dataset that captures complex user requirements using SQL statements. |
| Outcome: | The proposed dataset captures complex, real-world user requirements. |
UNCERTAINTY-LINE: Length-Invariant Estimation of Uncertainty for Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate misleading or outright incorrect information. |
| Approach: | They propose a method that debiases uncertainty scores on output length and uses residuals as corrected, length-invariant estimates. |
| Outcome: | The proposed method improves over nominally length-normalized methods on machine translation, summarization, and question-answering tasks. |
CERET: Cost-Effective Extrinsic Refinement for Text Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate incomplete, biased or misleading outputs in their initial attempts. |
| Approach: | They propose a method for refining text generation that takes into account semantic stability, entailment and inter-sample uncertainty measures. |
| Outcome: | The proposed method outperforms self-consistency and self-rerank baselines under various task setups by 1.6% and 3.5% respectively. |
LINKAGE: Listwise Ranking among Varied-Quality References for Non-Factoid QA Evaluation via LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Non-factoid (NF) question answering is challenging to evaluate due to diverse potential answers and no objective criterion. |
| Approach: | They propose a listwise NFQA evaluation approach that uses Large Language Models to rank candidate answers in a descending list of reference answers sorted by descending quality. |
| Outcome: | The proposed method has higher correlations with human annotations than standard methods. |
ZEBRA: Leveraging Model-Behavioral Knowledge for Zero-Annotation Preference Dataset Construction (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent efforts in LLM alignment focus on instance-wise supervision, costing substantial . ZEBRA binarizes response pairs by evaluating the quality and similarity of their origin models . |
| Approach: | They propose a model behavior-wise zero-annotation framework that binarizes preference data . ZEBRA binarized response pairs by evaluating the quality and similarity of their origin models . |
| Outcome: | The proposed framework achieves comparable alignment performance to instance-supervised methods . |
Semi-Automated Construction of Sense-Annotated Datasets for Practically Any Language (2025.coling-main)
Copied to clipboard
Jai Riley, Bradley M. Hauer, Nafisa Sadaf Hriti, Guoqing Luo, Amir Reza Mirzaei, Ali Rafiei, Hadi Sheikhi, Mahvash Siavashpour, Mohammad Tavakoli, Ning Shi, Grzegorz Kondrak
| Challenge: | Word sense disambiguation is a widely studied NLP task of identifying the meaning of a word in context. |
| Approach: | They propose a method to create parallel sense-annotated datasets in English . they use machine translation, word alignment, sense projection, and sense filtering to produce silver annotations . |
| Outcome: | The proposed method produces parallel sense-annotated datasets on Farsi, Chinese, and Bengali . the results are higher than those obtained with recent multilingual systems, the authors say . |
AESOP: Paraphrase Generation with Adaptive Syntactic Control (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for paraphrase generation use fixed syntactic structures for all input sentences. |
| Approach: | They propose to add syntactical control to a pretrained language model to generate fluent paraphrases using a retrieval-based selection module. |
| Outcome: | The proposed model achieves state-of-the-art on semantic preservation and syntactic conformation on two benchmark datasets with ground-truth syntaktic control from human-annotated exemplars. |
The Future is not One-dimensional: Complex Event Schema Induction by Graph Modeling for Event Prediction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Event schemas encode knowledge of stereotypical structures of events and their connections . previous work on event schema induction focuses on atomic events or linear temporal sequences . |
| Approach: | They propose a Temporal Complex Event Schema: a graph-based schema representation that encompasses events, arguments, temporal connections and argument relations. |
| Outcome: | The proposed model outperforms existing models on HITS@1 by 17.8%. |
KNOT: Knowledge Distillation Using Optimal Transport for Solving NLP Tasks (2022.coling-1)
Copied to clipboard
| Challenge: | Knowledge Distillation using Optimal Transport (KNOT) aims to distill the natural language semantic knowledge from multiple teacher networks to a student network. |
| Approach: | They propose to distill natural language semantic knowledge from multiple teacher networks to a student network by learning to minimize the optimal transport cost of its assigned probability distribution over the labels to the weighted sum of probabilities predicted by the (local) teacher models. |
| Outcome: | The proposed method shows improvements in the global model’s SD performance over the baseline across three NLP tasks while performing on par with Entropy-based distillation on standard accuracy and F1 metrics. |
COPA-SSE: Semi-structured Explanations for Commonsense Reasoning (2022.lrec-1)
Copied to clipboard
| Challenge: | Semi-structured explanations for Choice of Plausible Alternatives (COPA-SSE) are a crowdsourced dataset of 9,747 common sense explanations . |
| Approach: | They propose a semi-structured approach to explain Choice of Plausible Alternatives questions using a crowdsourced dataset of 9,747 common sense explanations with ConceptNet relations but freely written concepts. |
| Outcome: | The proposed explanations are geared towards commonsense reasoners operating on knowledge graphs and serve as a starting point for improving such systems. |
SuperWriter: Reflection-Driven Long-Form Generation with Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Long-form text generation remains a challenge for large language models . generating extended sequences often leads to degraded coherence and logical consistency . |
| Approach: | They propose a framework that integrates explicit structured thinking into long-form text generation. |
| Outcome: | The proposed framework surpasses even larger-scale models in evaluation and human evaluation. |
SynDARin: Synthesising Datasets for Automated Reasoning in Low-Resource Languages (2025.coling-main)
Copied to clipboard
| Challenge: | Question Answering datasets are scarce for languages other than English due to the cost and difficulties of collection and manual annotation. |
| Approach: | They propose a method for generating and validating QA datasets for low-resource languages . they use English data as context to generate synthetic multiple-choice (MC) question-answer pairs . |
| Outcome: | The proposed method maintains quality, reduces likelihood of factual errors, and circumvents costly annotation. |
Data-Efficient Concept Extraction from Pre-trained Language Models for Commonsense Explanation Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to extract concepts from pre-trained language models are not suitable for commonsense explanation generation. |
| Approach: | They propose a method to extract the key explanation concept from pre-trained language models by fine-tuning it with 20% training data and using a metric to evaluate the retrieved concepts. |
| Outcome: | The proposed method improves evaluation metrics over pre-trained language models and the existing models. |
FastDiff 2: Revisiting and Incorporating GANs and Diffusion Models in High-Fidelity Speech Synthesis (2023.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that Generative adversarial networks sacrifice sample diversity for quality and speed, while diffusion models exhibit outperformed sample quality and diversity at a high computational cost. |
| Approach: | They propose to combine GANs and diffusion probabilistic models to achieve better sample quality and diversity. |
| Outcome: | The proposed models outperform GANs and diffusion models in speech synthesis . the proposed models enjoy an efficient 4-step sampling process and exhibit better sample diversity . |
CDS: Data Synthesis Method Guided by Cognitive Diagnosis Theory (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse tasks, but the complexity of emerging tasks and higher performance demands highlight the need for continuous improvement. |
| Approach: | They propose a method that refines evaluation results and characterizes model profiles at the knowledge component level. |
| Outcome: | The proposed method improves performance across multiple benchmarks and academic exams. |
An Empirical Study of In-context Learning in LLMs for Machine Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies focus on optimizing translation quality, with limited attention to understanding specific aspects of ICL that influence the said quality. |
| Approach: | They conduct the first of its kind, exhaustive study of in-context learning for machine translation (MT) they establish that ICL is primarily example-driven and not instruction-driven . |
| Outcome: | The proposed model is based on examples and not instruction-driven learning. |
Learning to Rank Visual Stories From Human Ranking Data (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies on visual storytelling (VIST) use automated evaluation metrics for text generation. |
| Approach: | They develop a Vrank metric that repurposes human evaluation results for automatic evaluation. |
| Outcome: | The proposed model is more accurate than existing metrics and is generalizable to textual stories. |
An In-depth Study on Internal Structure of Chinese Words (2021.acl-long)
Copied to clipboard
Chen Gong, Saihao Huang, Houquan Zhou, Zhenghua Li, Min Zhang, Zhefeng Wang, Baoxing Huai, Nicholas Jing Yuan
| Challenge: | Unlike English letters, Chinese characters have rich and specific meanings. |
| Approach: | They propose to model Chinese words' internal structures as dependency trees with 11 labels for distinguishing syntactic relationships. |
| Outcome: | The proposed model of Chinese word-internal structures shows it can be used to parse sentences . it shows that the model can be applied to a sentence-level task with a competitive dependency parser. |
Enhancing Argument Summarization: Prioritizing Exhaustiveness in Key Point Generation and Introducing an Automatic Coverage Evaluation Metric (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for summarizing arguments are incapable of distinguishing between generated key points of different qualities. |
| Approach: | They propose an extractive approach that generates concise, high quality key points . they propose to use a clustering approach to generate key points from raw arguments . |
| Outcome: | The proposed method outperforms state-of-the-art methods for key point generation . it offers concise, high quality generated key points with higher coverage of reference summaries . |
Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have explored using LLMs for efficient data collection. |
| Approach: | They propose a method that takes into account the characteristics of the desired dataset and monitors the status of the generated data. |
| Outcome: | The proposed method improves safety and quality of three representative large language models against safety issues without sacrificing model utility. |
Improving Both Domain Robustness and Domain Adaptability in Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to domain adaptation for NMT depend on high-quality parallel data. |
| Approach: | They propose a meta-learning framework which improves domain robustness and adaptability . they use a word-level domain mixing model and a domain classifier to integrate it . |
| Outcome: | The proposed approach improves domain robustness and adaptability in seen and unseen domains. |
Lost in Inference: Rediscovering the Role of Natural Language Inference for Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | In the recent past, a popular way of evaluating natural language understanding was to consider a model’s ability to perform natural language inference (NLI) tasks. |
| Approach: | They focus on five different NLI benchmarks across six models of different scales and examine how their accuracies develop during training. |
| Outcome: | The softmax distributions of models align with human label distributions in cases where statements are ambiguous or vague. |
A Semi-Automated Live Interlingual Communication Workflow Featuring Intralingual Respeaking: Evaluation and Benchmarking (2022.lrec-1)
Copied to clipboard
| Challenge: | Traditionally, live interlingual communication has been achieved only with the help of human interpreters. |
| Approach: | They propose a semi-automated workflow which uses a human respeaker and speaker-dependent speech recognition software to deliver punctuated same-language output of superior quality than the out-of-the-box ASR system. |
| Outcome: | The proposed workflow produces a similar quality output to the best-in-class simultaneous interpreters working with the same source speeches from the European Parliament. |
Literary Machine Translation under the Magnifying Glass: Assessing the Quality of an NMT-Translated Detective Novel on Document Level (2020.lrec-1)
Copied to clipboard
| Challenge: | Several studies have demonstrated that translation quality has improved enormously since the emergence of neural machine translation systems. |
| Approach: | They performed a document-level evaluation of the raw NMT output of an entire novel and annotated it in two steps: first all fluency errors, then all accuracy errors. |
| Outcome: | The results show that translation quality has improved enormously since the emergence of neural machine translation systems. |
Leveraging Multiple Teachers for Test-Time Adaptation of Language-Guided Classifiers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent approaches focus on language-guided classifiers that can generalize in zero-shot settings, but their performance varies significantly between different language explanations in unpredictable ways. |
| Approach: | They propose a framework that uses data programming to adapt a language-guided classifier for a new task when provided with multiple teachers and unlabeled test examples. |
| Outcome: | The proposed framework outperforms a baseline from previous work by 9.3%. |
Two Directions for Clinical Data Generation with Large Language Models: Data-to-Label and Label-to-Data (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) can generate natural language texts for various domains and tasks, but their potential for clinical text mining is under-explored. |
| Approach: | They propose a pragmatic taxonomy for AD sign and symptom progression based on expert knowledge and train a system to detect AD-related signs and symptoms from EHRs. |
| Outcome: | The proposed taxonomy outperforms existing methods using only the gold dataset and silver datasets. |
CQG: A Simple and Effective Controlled Generation Framework for Multi-hop Question Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Current models can not ensure the complexity of generated questions, so they may generate shallow questions that can be answered without multi-hop reasoning. |
| Approach: | They propose a controlled framework to generate multi-hop questions that contain key entities in multi- hop reasoning chains and a novel Transformer-based decoder to guarantee that key entities appear in the questions. |
| Outcome: | The proposed model outperforms the state-of-the-art model 25% on HotpotQA. |
Incorporating Semantic Attention in Video Description Generation (L18-1)
Copied to clipboard
| Challenge: | Existing methods to generate video descriptions fail to mention objects and actions in videos . |
| Approach: | They propose an LSTM-based sequence-to-sequence model with semantic attention mechanism for video description generation that includes external fine-grained visual information. |
| Outcome: | The proposed model can selectively focus on external fine-grained visual information and have a better quality of video descriptions. |
Synthesizing Conversations from Unlabeled Documents using Automatic Response Segmentation (2024.findings-acl)
Copied to clipboard
| Challenge: | Several datasets have been developed for building conversational question answering systems. |
| Approach: | They propose a robust dialog synthesising method that learns segmentation instead of using sentence boundaries. |
| Outcome: | The proposed method achieves superior quality when compared to WikiDialog . it also improves performance across OR-QuAC benchmarks . |
Wikinflection Corpus: A (Better) Multilingual, Morpheme-Annotated Inflectional Corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | Inflectional corpora with annotated morpheme boundaries are scarce in the NLP community . a generated, multilingual inflectional lexicon with morphological features is not as good as UniMorph's . |
| Approach: | They evaluate a multilingual inflectional corpus with morpheme boundaries from the English Wiktionary and the UniMorph project's inflection corpus. |
| Outcome: | The generated Wikinflection corpus is not as good as UniMorph's, but extracts significant amount of words from the intersection of the two corpora. |
Quantifying the Dialect Gap and its Correlates Across Languages (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Historically, studies investigating minority variants of languages have been limited to a select few languages. |
| Approach: | They evaluate state-of-the-art large language models for regional dialects of several high- and low-resource languages and analyze how regional dialect gap is correlated with economic, social, and linguistic factors. |
| Outcome: | The proposed model is compared with two high-use applications and shows that it can solve the regional dialect gap. |
Getting More Data for Low-resource Morphological Inflection: Language Models and Data Augmentation (2020.lrec-1)
Copied to clipboard
| Challenge: | Morphological inflection is the process that generates the word form given its lexeme and morphological properties. |
| Approach: | They propose to use language models and data augmentation to improve morphological inflection without annotating more data. |
| Outcome: | The proposed model improves by 1.5% with the langauge model and by 9% with the data augmentation. |
Does Corpus Quality Really Matter for Low-Resource Languages? (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on multilingual pre-training has relied on automatically filtered versions of CommonCrawl. |
| Approach: | They propose to use tailored crawling to identify and scrape websites with high-quality content to improve representation learning in Basque. |
| Outcome: | The proposed corpus, called EusCrawl, has a much higher quality according to native annotators than the Basque portion of popular multilingual corpora like CC100 and mC4. |
Controllable Open-ended Question Generation with A New Question Type Ontology (2021.acl-long)
Copied to clipboard
| Challenge: | Existing question types are limited to generating multiple-sense questions . we present a question type-aware question generation framework to generate open-ended questions based on multiple-phrase questions - a task that is less explored . |
| Approach: | They propose a question type-aware question generation framework which predicts question focuses and produces the question. |
| Outcome: | The proposed model improves question quality over competitive comparisons on large-scale datasets. |
UniSent: Universal Adaptable Sentiment Lexica for 1000+ Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | Sentiment lexica are vital for sentiment analysis in absence of document-level annotations . linguistic resources are limited for at least a few hundred languages, putting them at risk of extinction . |
| Approach: | They introduce UniSent universal sentiment lexica for 1000+ languages . they use a Bible corpus to project sentiment information from English to other languages based on Twitter data . |
| Outcome: | The proposed method mitigates domain mismatch between Bible and Twitter by using embeddings . it compares to other sentiment seeding methods in a subset of languages with ground truth available . |
Scalable and Domain-General Abstractive Proposition Segmentation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Several recent studies have demonstrated the utility of proposition segmentation for downstream tasks. |
| Approach: | They propose a scalable, yet accurate, proposition segmentation model that can be supervised by LLMs. |
| Outcome: | The proposed model improves on training on annotated datasets and shows that it is easy to use. |
PRISM: Efficient Long-Range Reasoning With Short-Context LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing solutions to long-range language tasks require large compute budgets and complex task-specific design choices. |
| Approach: | They propose an in-context method that uses structured schemas to generate short-contemporary outputs. |
| Outcome: | a new in-context method outperforms baselines on diverse tasks with 4x shorter contexts . it scales down to tiny contexts without increasing costs or sacrificing quality . |
Forecasting Future International Events: A Reliable Dataset for Text-Based Event Modeling (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches for text-based event prediction are limited in quality due to dynamic nature of international relations and conflicting economic dynamics. |
| Approach: | They propose a novel dataset that leverages the advanced reasoning capabilities of large-language models to address these limitations. |
| Outcome: | The proposed dataset features high-quality scoring labels generated through advanced prompt modeling and rigorously validated by domain experts in political science. |
Improving Machine Translation of Educational Content via Crowdsourcing (L18-1)
Copied to clipboard
Maximiliana Behnke, Antonio Valerio Miceli Barone, Rico Sennrich, Vilelmini Sosoni, Thanasis Naskos, Eirini Takoulidou, Maria Stasimioti, Menno van Zaanen, Sheila Castilho, Federico Gaspari, Panayota Georgakopoulou, Valia Kordoni, Markus Egg, Katia Lida Kermanidis
| Challenge: | Using crowdsourcing to train neural machine translation models is expensive and expensive . professional outsourcing of bilingual data is expensive if the translations are of a lower quality . |
| Approach: | They analyze the impact of crowdsourcing on the quality of in-domain training data . they use translations of MOOCs from English to eleven languages to fine-tune machine translation models . |
| Outcome: | The proposed method improves on general-domain training data and with pre-existing in-domain corpora. |
Paraphrase Generation by Learning How to Edit from Samples (2020.acl-main)
Copied to clipboard
| Challenge: | Experimental results show the superiority of our retrieval-based paraphrase generation model in terms of both automatic metrics and human evaluation of relevance, grammaticality, and diversity of generated paraphrases. |
| Approach: | They propose a retrieval-based method for paraphrase generation which uses a novel editor module to extract edits from paraphrase pairs. |
| Outcome: | The proposed model outperforms existing models in automatic metrics and human evaluation of relevance, grammaticality, and diversity of generated paraphrases. |
KMI: A Dataset of Korean Motivational Interviewing Dialogues for Psychotherapy (2025.naacl-long)
Copied to clipboard
| Challenge: | Motivational Interviewing (MI) is gaining attention as a theoretical basis for mental health chatbots. |
| Approach: | They propose a framework that simulates MI sessions enriched with the expertise of professional therapists by using large language models to generate utterances through prompt engineering. |
| Outcome: | The proposed framework simulates MI sessions enriched with the expertise of professional therapists and employs large language models to generate utterances through prompt engineering. |
Dialogizer: Context-aware Conversational-QA Dataset Generation from Textual Sources (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing dialog inpainting methods generate ConvQA datasets with low contextual relevance due to insufficient learning of question-answer alignment. |
| Approach: | They propose a dialog inpainting method that generates ConvQA datasets from documents . they propose re-ranking tasks and a framework that generate contextually relevant questions . |
| Outcome: | The proposed framework generates ConvQA datasets with high contextual relevance from textual sources. |
Can Many-Shot In-Context Learning Help LLMs as Evaluators? A Preliminary Empirical Study (2025.coling-main)
Copied to clipboard
| Challenge: | Existing evaluation approaches to evaluate Large Language Models are affected by potential biases within LLMs. |
| Approach: | They propose two many-shot In-Context Learning (ICL) prompt templates to help LLM evaluators mitigate potential biases. |
| Outcome: | The proposed templates reduce biases by using in-context examples with model-generated rationales as references. |
Into the Unknown Unknowns: Engaged Human Learning through Participation in Language Model Agent Conversations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in language models (LMs) and retrieval-augmented generation (RAG) have led to more capable chatbots and generative search engines. |
| Approach: | They propose to emulate the educational scenario where children/students learn by listening to and participating in conversations of their parents/teachers by watching and steering the discourse among several LM agents. |
| Outcome: | The proposed system outperforms baseline methods on discourse trace and report quality and is preferred by 70% of participants over a search engine and 78% over sabota. |
Taking a turn for the better: Conversation redirection throughout the course of mental-health therapy (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Mental-health therapy involves a complex conversation flow in which patients and therapists continuously negotiate what should be talked about next. |
| Approach: | They propose a measure to quantify the extent to which a certain utterance immediately redirects the flow of the conversation in a large, widely-used online therapy platform. |
| Outcome: | The proposed measure measures the extent to which a certain utterance immediately redirects the flow of the conversation over multiple sessions in a large, widely-used online therapy platform. |
Creating a Dataset for Named Entity Recognition in the Archaeology Domain (2020.lrec-1)
Copied to clipboard
| Challenge: | Currently, there is no way to find 'by-catch', single finds of a different type, in the metadata of excavation reports. |
| Approach: | They propose to train NER classifiers on Dutch excavation reports to help archaeologists find structured information in archaic documents. |
| Outcome: | The proposed dataset contains 31k annotations between six entity types (artefact, time period, place, context, species & material). |
Sarcasm-R1: Enhancing Sarcasm Detection through Focused Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for sarcasm detection are limited by supervised learning or prompt engineering . a new approach decomposes sarcasm detection into three dimensions: language, context, and emotion . |
| Approach: | They propose a method that decomposes sarcasm detection into three dimensions: language, context, and emotion. |
| Outcome: | The proposed method outperforms state-of-the-art methods in most cases. |
Arab Voices: Mapping Standard and Dialectal Arabic Speech Technology (2026.findings-acl)
Copied to clipboard
| Challenge: | Dialectal Arabic datasets embody a range of domain, dialect, and quality. |
| Approach: | They propose a framework for automatic speech recognition in dialectal Arabic to address the limited data availability encountered in dialects. |
| Outcome: | The proposed framework provides access to 31 datasets covering 14 dialects to better address the limited data availability encountered in dialectal Arabic speech processing. |
Book2Dial: Generating Teacher Student Interactions from Textbooks for Cost-Effective Development of Educational Chatbots (2024.findings-acl)
Copied to clipboard
| Challenge: | Educational chatbots are a promising tool for assisting student learning, but high-quality data is difficult to obtain due to privacy concerns. |
| Approach: | They propose a framework for generating synthetic teacher-student interactions grounded in a set of textbooks and propose to open-source their results. |
| Outcome: | The proposed framework captures a key aspect of learning interactions where curious students with partial knowledge ask teachers questions about the material in the textbook. |
Split or Merge: Which is Better for Unsupervised RST Parsing? (D19-1)
Copied to clipboard
| Challenge: | Rhetorical Structure Theory (RST) parsers have been based on supervised learning approaches that require an annotated corpus of sufficient size and quality. |
| Approach: | They propose two unsupervised methods that build an optimal RST tree based on a dissimilarity score function for splitting a text span into smaller ones and a similarity score for merging two adjacent spans into a large one. |
| Outcome: | The proposed method achieves the best score on English and German RST treebanks, around 0.8 F1 score, close to the previous supervised parsers. |
Quantifying the Influence of Evaluation Aspects on Long-Form Response Assessment (2025.coling-main)
Copied to clipboard
| Challenge: | Evaluating the outputs of large language models (LLMs) on long-form generative tasks remains challenging. |
| Approach: | They propose to compute an overall quality score as a weighted average of factuality, informative-ness, and formality as compared to previous metrics. |
| Outcome: | The proposed method achieves stronger correlations with human judgments compared to previous metrics. |
Rethinking Masked Language Modeling for Chinese Spelling Correction (2023.acl-long)
Copied to clipboard
| Challenge: | Existing CSC models over-fit the error model while under-fitting the language model, resulting in poor generalization to out-of-distribution error patterns. |
| Approach: | They propose to use a multi-domain benchmark LEMON to assess the open domain generalization of Chinese Spelling Correction models. |
| Outcome: | The proposed method achieves state-of-the-art results on SIGHAN, ECSpell, and LEMON. |
DIVE: Towards Descriptive and Diverse Visual Commonsense Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Towards human-level visual understanding, visual commonsense generation has been introduced . but current research on visual commonense generation ignores an important human cognitive ability . |
| Approach: | They propose a visual commonsense generation framework to improve inferences by visual common sense generation. |
| Outcome: | The proposed framework outperforms state-of-the-art models in descriptiveness and diversity . human evaluations confirm that the framework aligns closely with human judgments on descriptiveness . |
Draft
& Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for accelerating Large Language Models have been criticized for their inference costs and inefficient decoding. |
| Approach: | They propose a self-speculative decoding approach for accelerating Large Language Models without an auxiliary model. |
| Outcome: | The proposed method achieves a speedup of up to 1.99 with no additional neural network training and no extra memory footprint. |
Redefining Machine Simultaneous Interpretation: From Incremental Translation to Human-Like Strategies (2026.findings-acl)
Copied to clipboard
| Challenge: | Simultaneous machine translation requires high-quality translations under strict real-time constraints. |
| Approach: | They extend the action space of simultaneous machine translation with four adaptive actions . they adapt these actions in a large language model framework and construct training references . |
| Outcome: | The proposed framework improves semantic metrics and achieves lower delay compared to reference translations and salami-based baselines. |
Proxy Indicators for the Quality of Open-domain Dialogues (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for evaluation of open-domain dialogues are expensive and require human annotators to evaluate their quality. |
| Approach: | They propose to use a deep-learning model trained on the general language understanding evaluation benchmark to serve as a quality indication of open-domain dialogues. |
| Outcome: | The proposed model can infer various quality metrics and derive a component-based overall score. |
Mapping Brains with Language Models: A Survey (2023.findings-acl)
Copied to clipboard
| Challenge: | accumulated evidence for brain and language model activations remains ambiguous, but correlations with model size and quality provide grounds for cautious optimism. |
| Approach: | They examine the evidence accumulated by 30 studies spanning 10 datasets and 8 metrics to determine whether there is any overlap between brain and language model activations. |
| Outcome: | The findings suggest that representations extracted from NLP models can (partially) explain the signal found in neural data. |
CN-AutoMIC: Distilling Chinese Commonsense Knowledge from Pretrained Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing commonsense knowledge graphs are limited to English, hindering research in non-English languages. |
| Approach: | They propose a Chinese CKG generated from multilingual PLMs that is translated into Chinese . they propose 'generate-by-category' strategy to reduce invalid generation . |
| Outcome: | The proposed CKG has high quality and diversity, surpassing the direct translation version of similar English CKGs. |
How to Engage your Readers? Generating Guiding Questions to Promote Active Reading (2024.acl-long)
Copied to clipboard
| Challenge: | Using questions in written text is an effective strategy to enhance readability, but what makes an active reading question good, what the linguistic role of these questions is, and what is their impact on human reading remains understudied. |
| Approach: | They present a dataset of 10K in-text questions from textbooks and scientific articles and explore various approaches to generate such questions using language models. |
| Outcome: | The generated questions are of high quality and are almost as effective as human-written questions in terms of improving readers’ memorization and comprehension. |
Contextual Interaction for Argument Post Quality Assessment (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for assessing the quality of natural language arguments are limited . existing methods focus on evaluating individual argument posts, but they often fail to distinguish between arguments with a narrow quality gap. |
| Approach: | They propose to use supervised contrastive learning to model arguments' quality . large language models with in-context examples harness the power of LLMs . |
| Outcome: | The proposed approach outperforms state-of-the-art models on a publicly available dataset . it shows that the LLMs with in-context examples are more effective than baseline models . |
CE-DA: Custom Embedding and Dynamic Aggregation for Zero-Shot Relation Extraction (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to predict relationships with given entity pairs are lacking in supervised methods. |
| Approach: | They propose a framework for zero-shot Relation Extraction that includes two modules: Custom Embedding and Dynamic Aggregation. |
| Outcome: | The proposed framework shows competitive performance on two ZSRE datasets. |
New Protocols and Negative Results for Textual Entailment Data Collection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Natural language inference data has proven useful in benchmarking and as pretraining data for tasks requiring language understanding. |
| Approach: | They propose four alternative protocols to improve annotation quality and diversity . they use 8.5k-example training sets to compare different protocols . |
| Outcome: | The proposed protocols improve the ease of training and quality of the examples. |
QGEval: Benchmarking Multi-dimensional Evaluation for Question Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing metrics fail to align well with human judgments when evaluating QG questions. |
| Approach: | They propose a multi-dimensional evaluation benchmark for QG and automatic metrics that evaluates questions and automated metrics across 7 dimensions. |
| Outcome: | The proposed benchmark evaluates QG models and automatic metrics across 7 dimensions . it shows that most QG model performs unsatisfactorily in terms of answerability and answer consistency . |
On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) provide a data-centric solution to alleviate limitations of real-world data with synthetic data generation. |
| Approach: | They propose a generic workflow for LLM-driven synthetic data generation. |
| Outcome: | The proposed workflows highlight gaps in existing research and outline avenues for future studies. |
A Framework for Evaluation of Machine Reading Comprehension Gold Standards (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing literature on machine reading comprehension (MRC) data is limited on the data design of gold standards. |
| Approach: | They propose a framework to investigate linguistic features, lexical cues and ambiguity in MRC gold standards. |
| Outcome: | The proposed framework investigates the present linguistic features, required reasoning and background knowledge and factual correctness on the one hand, and the presence of lexical cues as a lower bound for the requirement of understanding on the other. |
Curating Datasets for Better Performance with Example Training Dynamics (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve data quality but rely on data quantity to improve performance are not effective. |
| Approach: | They propose a method for weighing the relative importance of examples in a dataset based on their Example Training dynamics (ETD) they propose an active learning approach for computing ETD during training rather than as a preprocessing step. |
| Outcome: | The proposed method can be used to improve performance in in-distribution and out-of-distortion testing. |
Parallel Refinements for Lexically Constrained Text Generation with BART (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work injects lexical constraints into the output, which generates generic or ungrammatical sentences and has high computational complexity. |
| Approach: | They propose a model that incorporates pre-specified keywords into the output to control the generated text. |
| Outcome: | The proposed model decomposes the generated text into two sub-tasks and improves the sentence quality. |
wav2vec-S: Adapting Pre-trained Speech Models for Streaming (2024.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained speech models have advanced speech-related tasks, including speech recognition and translation. |
| Approach: | They propose a pre-trained speech model that incorporates modifications to ensure consistent speech representations during training and inference phases for streaming speech inputs. |
| Outcome: | The proposed model outperforms baseline models on speech recognition and translation tasks and achieves a superior balance between quality and latency. |
In Neural Machine Translation, What Does Transfer Learning Transfer? (2020.acl-main)
Copied to clipboard
| Challenge: | a recent study found that word embeddings are not necessary for transfer learning. |
| Approach: | They perform several ablation studies that limit information transfer and measure the quality impact across three language pairs to gain a black-box understanding of transfer learning. |
| Outcome: | The proposed method can eliminate the need for a warm-up phase when training transformer models in high resource language pairs. |
Efficient Entity Candidate Generation for Low-Resource Languages (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing approaches for cross-lingual entity linking are not suitable for English. |
| Approach: | They propose a candidate generation problem in cross-lingual entity linking with a focus on low-resource languages. |
| Outcome: | The proposed solution outperforms the state-of-the-art approach on 9 real-world datasets and query types. |
Best-k Search Algorithm for Neural Text Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Modern natural language generation paradigms require a decoding strategy to obtain quality sequences out of the model. |
| Approach: | They propose a deterministic search algorithm balancing quality and diversity . they investigate the vanilla best-first search algorithm and propose k-k search algorithm. |
| Outcome: | The proposed algorithm is parameter-free, lightweight, efficient, and easy-to-use. |
The Linguistic Category Model in Polish (LCM-PL) (L18-1)
Copied to clipboard
| Challenge: | a new version of the Linguistic Category Model (LCM) dictionary for the Polish language is available for use and integrates with the Polish WordNet. |
| Approach: | They propose to use a dictionary that is annotated manually in its most important parts . they propose to add more manually annotating senses and increase quality of automated annotations . |
| Outcome: | The proposed dictionary is the first widely usable version of the resource . it will have more manually annotated senses and more automated annotations . |
Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a new approach to generate adversarial data is needed to improve question answering models . crowdworkers can fool a model only 8.8% of the time, compared to 17.6% for a trained model without synthetic data. |
| Approach: | They develop a pipeline that generates questions and then filters or labels them to improve quality. |
| Outcome: | The proposed approach improves state-of-the-art on a human-written adversarial dataset by 3.7F1 and improves model generalisation on nine of the twelve MRQA datasets. |
Neural CRF Model for Sentence Alignment in Text Simplification (2020.acl-main)
Copied to clipboard
| Challenge: | Text simplification systems are based on the quality and quantity of complex-simple sentence pairs extracted by aligning sentences between parallel articles. |
| Approach: | They propose a neural CRF alignment model which leverages the sequential nature of sentences in parallel documents and utilizes a sentence pair model to capture semantic similarity. |
| Outcome: | The proposed model outperforms previous work on monolingual sentence alignment task by more than 5 points in F1. |
G2: Guided Generation for Enhanced Output Diversity in LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to enhance output diversity but compromise quality of outputs. |
| Approach: | They propose a training-free plug-and-play method that enhances output diversity while preserving generation quality. |
| Outcome: | The proposed method enhances output diversity while maintaining an optimal balance between diversity and quality. |
Reflect, Not Reflex: Inference-Based Common Ground Improves Dialogue Response Quality (2022.emnlp-main)
Copied to clipboard
| Challenge: | Currently, human communication models fail to explicitly model common ground (CG) . less than half of the responses in current data is rated as high quality . |
| Approach: | They propose a dataset that annotates dialogues with explicit CG and solicits 9k diverse responses each following one common ground. |
| Outcome: | The proposed dataset annotates dialogues with explicit CG and solicits 9k diverse responses each following one common ground. |
Large-Scale Multimodal Knowledge Graph about Classical Chinese Poetry: Fine-grained Method and Comprehensive Evaluation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on classical Chinese poetry are limited by modality constraints, dataset size, or the level of refinement. |
| Approach: | They propose to construct a large-scale and fine-grained multimodal knowledge graph of classical Chinese poetry using an informative ontology graph and a text-image alignment method. |
| Outcome: | The proposed method collects knowledge about classical Chinese poetry from ontology graphs and performs four tasks that demonstrate its comprehensiveness and high quality. |
EmoTransKG: An Innovative Emotion Knowledge Graph to Reveal Emotion Transformation (2024.findings-acl)
Copied to clipboard
| Challenge: | EmoTransKG establishes connections and transformations between emotions across open-textual events. |
| Approach: | They propose an Emotion Knowledge Graph that establishes connections and transformations between emotions across diverse open-textual events. |
| Outcome: | The proposed model integrates with existing conversational emotion recognition models to improve the quality and effectiveness of EmoTransKG. |
LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Texts (2024.acl-long)
Copied to clipboard
| Challenge: | Existing frameworks for the automated evaluation of natural language texts are based on a large language model (LLM) that fails to agree with human judges and is not fully validated by the human judges. |
| Approach: | They propose a large language model (LLM) that generates a distribution over potential responses to assess multiple dimensions of interest. |
| Outcome: | The proposed framework predicts human judges' assessment of user satisfaction on a scale of 1–4 with an RMS error 0.5, a 2 improvement over the uncalibrated baseline. |
Empowering Large Language Models for Textual Data Augmentation (2024.findings-acl)
Copied to clipboard
| Challenge: | True. True. False |
| Approach: | False slants are proposed to generate a large pool of augmentation instructions and select the most suitable task-informed instructions. |
| Outcome: | False omissions: the proposed approach consistently generates augmented data with better quality compared to non-LLM and LLM-based data augmentation methods. |
Benchmarking LLMs on Authentic Cases from Medical Journals (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing medical benchmarks suffer from performance saturation due to medical exam questions. |
| Approach: | They evaluate the performance of over 20 open-source and proprietary large language models and benchmark them against human medical experts. |
| Outcome: | The new benchmark is based on authentic clinical cases sourced from medical journals and implements rigorous human review process to ensure the quality and reliability of the benchmark. |
ArgAnalysis35K : A large-scale dataset for Argument Quality Analysis (2023.acl-long)
Copied to clipboard
| Challenge: | Existing datasets in argument quality detection lack quality, quantity and diversity of topics and arguments. |
| Approach: | They propose a dataset that adds a detailed explanation of why the argument made is true, applicable or impactful. |
| Outcome: | The proposed dataset covers 34,890 high-quality argument-analysis pairs and is the largest of its kind to our knowledge. |
DataSeer: A Manager-Centric Collaborative Multi-Agent Framework with Multi-Branch Reasoning for Automated Insight Discovery (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for automated insight discovery lack contextual coherence and coverage due to single-path exploration. |
| Approach: | They propose a Manager-Centric Collaborative Framework that integrates planner and executor . it ensures cross-episode contextual coherence and allows for adaptive sub-goal generation . |
| Outcome: | The proposed framework outperforms baselines on InsightBench and Inseval. |
Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on cross-lingual transferability of multilingual LMs show that they can perform tasks in low-resource languages. |
| Approach: | They propose a method to regularize the model from learning language invariant representations and a way to select model checkpoints without a development set in the target language. |
| Outcome: | The proposed method reduces the accidental translation problem by 68% and improves the ROUGE-L score by 1.5 on average. |
CEASR: A Corpus for Evaluating Automatic Speech Recognition (2020.lrec-1)
Copied to clipboard
Malgorzata Anna Ulasik, Manuela Hürlimann, Fabian Germann, Esin Gedik, Fernando Benites, Mark Cieliebak
| Challenge: | Automatic Speech Recognition (ASR) systems are increasingly needed for research and practical applications. |
| Approach: | They propose to use public speech corpora to evaluate the quality of automatic speech recognition (ASR) they calculate an average Word Error Rate (WER) per corpus, per system and per corpor-system pair . |
| Outcome: | The proposed corpus evaluates the quality of automatic speech recognition systems using public speech corpora and transcripts generated by state-of-the-art systems. |
Neural Machine Translation Quality and Post-Editing Performance (2021.emnlp-main)
Copied to clipboard
| Challenge: | a recent study has shown that MT post-editing can reduce translation quality and speed . a large-scale study involving 30 professional translators examined the relationship between MT performance and post-edited outputs. |
| Approach: | They examine the relationship between MT performance and post-editing time and quality . they use neural MT of high quality to improve translation quality based on phrase-based MT . |
| Outcome: | The proposed model is not stable predictor of time or quality, the authors say . they find that better MT systems lead to fewer changes in the sentences . |
QEFT: Quantization for Efficient Fine-Tuning of LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to optimize inference and fine-tuning for large language models have failed to improve all aspects of the process. |
| Approach: | They propose a new technique that accelerates both inference and fine-tuning while using fewer resources. |
| Outcome: | The proposed technique accelerates both inference and fine-tuning while using fewer resources. |
Large-Scale Diverse Synthesis for Mid-Training (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing data synthesis methods generate simplistic and homogeneous QA pairs with limited scale and diversity. |
| Approach: | They propose a framework to synthesize large-scale, diverse, and high-quality QA data for mid-training. |
| Outcome: | The proposed framework improves on 500B-token BoostQA data over pre-training benchmarks. |
Global Eye: Breaking the “Fixed Thinking Pattern” during the Instruction Expansion Process (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods focus on constructing multi-perspective prompts to expand instructions, overlooking the “Fixed Thinking Pattern” issue of Large Language Models. |
| Approach: | They propose a method that analyzes the statistical characteristics of newly generated instructions and updates the prompts after a fixed number of instruction expansions. |
| Outcome: | The proposed method surpasses open-source LLMs and GPT3.5 in several metrics. |
Language Models in Dialogue: Conversational Maxims for Human-AI Interactions (2024.findings-emnlp)
Copied to clipboard
Erik Miehling, Manish Nagireddy, Prasanna Sattigeri, Elizabeth Daly, David Piorkowski, John Richards
| Challenge: | Modern language models exhibit some inherent shortcomings, particularly in conversational settings. |
| Approach: | They propose a set of maxims for describing effective human-AI conversation that include quantity, quality, relevance, manner, benevolence, and transparency. |
| Outcome: | The proposed maxims are applied to human-AI interactions and are based on extensive research from the social science and AI communities. |
KazQAD: Kazakh Open-Domain Question Answering Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | KazQAD contains just under 6,000 unique questions with extracted short answers and nearly 12,000 passage-level relevance judgements. |
| Approach: | They introduce a Kazakh open-domain question answering dataset that can be used in reading comprehension and full ODQA settings. |
| Outcome: | The proposed dataset can be used in reading comprehension and full ODQA settings, as well as for information retrieval experiments. |
Review-Instruct: A Review-Driven Multi-Turn Conversations Generation Method for Large Language Models (2025.findings-acl)
Copied to clipboard
Jiangxu Wu, Cong Wang, Tianhuang Su, Lin Haozhi, JunYang JunYang, Zhangchao Zhangchao, Binqiang Pan, SongpanYang SongpanYang, Mingpeng Mingpeng, Kai Shi, Zixian Li
| Challenge: | Existing methods for generating multi-turn dialogue data struggle to ensure both diversity and quality in instructions. |
| Approach: | They propose a framework that synthesizes multi-turn conversations through an iterative "Ask-Respond-Review" process involving three agent roles: a Candidate, multiple Reviewers, and a Chairman. |
| Outcome: | The proposed framework synthesizes multi-turn conversations through an iterative "Ask-Respond-Review" process involving three agent roles: a Candidate, multiple Reviewers, and a Chairman. |
An Expanded Massive Multilingual Dataset for High-Performance Language Technologies (HPLT) (2025.acl-long)
Copied to clipboard
Laurie Burchell, Ona De Gibert Bonet, Nikolay Arefyev, Mikko Aulamo, Marta Bañón, Pinzhen Chen, Mariia Fedorova, Liane Guillou, Barry Haddow, Jan Hajič, Jindřich Helcl, Erik Henriksson, Mateusz Klimaszewski, Ville Komulainen, Andrey Kutuzov, Joona Kytöniemi, Veronika Laippala, Petter Mæhlum, Bhavitvya Malik, Farrokh Mehryary, Vladislav Mikhailov, Nikita Moghe, Amanda Myntti, Dayyán O’Brien, Stephan Oepen, Proyag Pal, Jousia Piha, Sampo Pyysalo, Gema Ramírez-Sánchez, David Samuel, Pavel Stepachev, Jörg Tiedemann, Dušan Variš, Tereza Vojtěchová, Jaume Zaragoza-Bernabeu
| Challenge: | a large number of textual data is needed to train state-of-the-art large language models. |
| Approach: | They propose a collection of monolingual and parallel corpora from the Internet Archive . they document the entire data pipeline and release the code to reproduce it . |
| Outcome: | The proposed collection of monolingual and parallel corpora is based on the HPLT v2 dataset . it includes 8T tokens covering 193 languages and 380M sentence pairs covering 51 languages . |
Know the Known and the Unknown: Reasonable Answer Generation with Knowledge-Informed Citations (2026.acl-long)
Copied to clipboard
Yichi Zhang, Zhuo Chen, Lingbing Guo, Jun Xu, Mengshu Sun, Zhizhen Liu, Lei Liang, Wen Zhang, Huajun Chen
| Challenge: | Existing approaches focus on generating multi-level citations linked to specific references, making it verifiable and trustworthy. |
| Approach: | They propose a new data construction pipeline and a benchmark to improve citation granularity and awareness of unknown information. |
| Outcome: | The proposed model improves on the existing benchmark and data construction pipeline and provides citation granularity and awareness of unknown information. |
SkillQG: Learning to Generate Question for Reading Comprehension Assessment (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing question generation systems focus on the literal nature of questions and rarely consider comprehension types of the generated questions. |
| Approach: | They propose a question generation framework with controllable comprehension types for machine reading comprehension models. |
| Outcome: | Empirical results show that SkillQG outperforms baselines in quality, relevance, and skill-controllability while showing a performance boost in downstream question answering task. |
Cross-Modal Attribute Insertions for Assessing the Robustness of Vision-and-Language Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to model multimodal data do not leverage cross-modal information . augmenting input text using cross-module attribute insertions results in poor performance . |
| Approach: | They propose a multimodal deep learning approach that adds visual attributes to inputs to enhance model robustness. |
| Outcome: | The proposed approach is modular, controllable, and task-agnostic. |
Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Reasoning Models (LRMs) often display unstable behaviors, e.g., hallucinating unsupported premises, overthinking simple tasks, and displaying higher sensitivity to prompt variations. |
| Approach: | They propose a graph-based analytical framework that clusters long, verbose CoT outputs into semantically coherent reasoning steps, then constructs directed reasoning graphs to capture contextual and logical dependencies among these steps. |
| Outcome: | The proposed framework enables quantitative evaluation of internal reasoning structure and quality beyond conventional metrics and provides practical insights for prompt engineering and cognitive analysis of LLMs. |
InstOptima: Evolutionary Multi-objective Instruction Optimization via Large Language Model-based Instruction Operators (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on automating instruction generation but do not consider other objectives that impact instruction quality. |
| Approach: | They propose an approach that treats instruction generation as an evolutionary multi-objective optimization problem. |
| Outcome: | The proposed approach improves fine-tuning performance and the generation of high-quality instructions. |
Finding Blind Spots in Evaluator LLMs with Interpretable Checklists (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models are increasingly relied upon to evaluate text outputs of other LLMs . however, concerns persist over the accuracy of these assessments and the potential for misleading conclusions. |
| Approach: | They propose a framework to assess the reliability of Large Language Models (LLMs) they propose ' FBI' framework to examine the proficiency of Evaluator LLMs in assessing four critical abilities . |
| Outcome: | The proposed framework assesses the performance of LLMs in text generation tasks. |
The Locality and Symmetry of Positional Encodings (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work uses fixed or learned positional encodings to infuse word order information into transformer-based language models. |
| Approach: | They conduct a systematic study of positional encodings in Bidirectional Masked Language Models. |
| Outcome: | The proposed model improves performance across a wide range of natural language understanding tasks. |
Reading Between the Prompts: How Stereotypes Shape LLM’s Implicit Personalization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has shown that such inferences can lead to lower quality responses for users assumed to be from minority groups. |
| Approach: | They analyze LLMs' latent user representations through both model internals and generated answers to targeted user questions. |
| Outcome: | The proposed models infer demographic attributes based on stereotypical signals, which persists even when the user explicitly identifies with a different demographic group. |
BAGELS: Benchmarking the Automated Generation and Extraction of Limitations from Scholarly Text (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a growing number of scientific publications have limitations as a source of uncertainty. |
| Approach: | They propose a computational architecture for extracting and generating limitations from scholarly papers using a novel Retrieval Augmented Generation technique. |
| Outcome: | The proposed architecture extracts limitations from ACL, NeurIPS, and PeerJ papers and supplementes them with external reviews. |
Re-Evaluating Evaluation for Multilingual Summarization (2024.emnlp-main)
Copied to clipboard
Jessica Forde, Ruochen Zhang, Lintang Sutawika, Alham Aji, Samuel Cahyawijaya, Genta Winata, Minghao Wu, Carsten Eickhoff, Stella Biderman, Ellie Pavlick
| Challenge: | Existing studies have shown that automated evaluation approaches correlate with human ratings in English, but this is unclear for other languages. |
| Approach: | They construct a small-scale pilot dataset containing article-summary pairs and human ratings in English, Chinese and Indonesian to measure the strength of summaries. |
| Outcome: | The results show that standard metrics are unreliable measures of quality in Chinese and Indonesian. |
PsychEval: A Multi-Session and Multi-Therapy Benchmark for High-Realism AI Psychological Counselor (2026.findings-acl)
Copied to clipboard
Qianjun Pan, Junyi Wang, Jie Zhou, Yutao Yang, Junsong Li, Kaiyin Xu, Yougen Zhou, Yihan Li, JingYuan Zhao, Qin Chen, Ningning Zhou, Kai Chen, Liang He
| Challenge: | Existing models focus on a single therapy, but complex cases require flexible strategies among various therapies. |
| Approach: | They propose a multi-session, multi-therapy, and highly realistic benchmark . it is designed to address three key challenges: 1) can we train a highly realistic AI counselor? 2) How to systematically evaluate an AI counselor?" |
| Outcome: | The proposed benchmark is annotated with extensive professional skills and includes over 677 meta-skills and 4577 atomic skills. |
Thoughts to Target: Enhance Planning for Target-driven Conversation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Empirical results demonstrate that our method significantly improves the planning ability of LLMs, especially in target-driven conversations. |
| Approach: | They propose a two-stage framework to improve the LLMs’ capability in planning conversations towards designated targets by distilling natural language plans from a target-driven conversation corpus and generating new plans with demonstration-guided in-context learning. |
| Outcome: | The proposed framework improves the ability of conversational models to plan towards designated targets and can be used to build extensive conversational AI. |
Aspect-based Sentiment Analysis via Synthetic Image Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in Aspect-Based Sentiment Analysis (ABSA) have shown promising results, yet the semantics derived solely from textual data remain limited. |
| Approach: | They propose a supervised image generation framework to generate synthetic images with alignment to text and sentiment information. |
| Outcome: | The proposed approach significantly outperforms state-of-the-art methods on multiple benchmark datasets. |
Rebalancing Label Distribution While Eliminating Inherent Waiting Time in Multi Label Active Learning Applied to Transformers (2024.lrec-main)
Copied to clipboard
| Challenge: | Data annotation is a resourceintensive endeavor, necessitating human involvement and expertise. |
| Approach: | They propose to annotate instances to rebalance label distribution by judiciously selecting and limiting the data to be annotated. |
| Outcome: | The proposed method mitigates biases, improves model performance and reduces strategy-dependent disparities. |
REFeREE: A REference-FREE Model-Based Metric for Text Simplification (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for text simplification lack a universal standard of quality and require a small number of human annotations. |
| Approach: | They propose to introduce a reference-free model-based metric with a 3-stage curriculum that can be applied to any quality standard with fewer annotations. |
| Outcome: | The proposed metric outperforms existing reference-based metrics in predicting ratings while requiring no reference simplifications at inference time. |
Local Normalization Distortion and the Thermodynamic Formalism of Decoding Strategies for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing decoding strategies for language models are based on heuristics and are difficult to improve. |
| Approach: | They propose a theory of decoding strategies for language models by expressing popular decoding algorithms as equilibrium states and stating objective functions they optimize. |
| Outcome: | The proposed method is based on ergodic theory and is difficult to improve . it shows that local normalization distortion is a fundamental defect of decoding strategies . |
TopXGen: Topic-Diverse Parallel Data Generation for Low-Resource Machine Translation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning and similarity search have been shown to improve LLMs' performance in machine translation, but they lag behind when dealing with low-resource languages. |
| Approach: | They propose a method that uses an LLM to generate topic-specific target-side data in the LRL. |
| Outcome: | The proposed approach boosts LLM translation performance during in-context learning and fine-tuning. |
STORM-BORN: A Challenging Mathematical Derivations Dataset Curated via a Human-in-the-Loop Multi-Agent Framework (2025.findings-acl)
Copied to clipboard
Wenhao Liu, Zhenyi Lu, Xinyu Hu, Jerry Zhang, Dailin Li, Jiacheng Cen, Huilin Cao, Haiteng Wang, Yuhan Li, Xie Kun, Dandan Li, Pei Zhang, Chengbo Zhang, Yuxiang Ren, Xiaohong Huang, Yan Ma
| Challenge: | Existing datasets suffer from outdated and insufficient challenging content, neglecting human-like reasoning, and limited reliability due to single-LLM generation. |
| Approach: | They propose a human-in-the-loop, multi-agent data generation framework that integrates reasoning-dense filters, multiagent collaboration, and human mathematicians’ evaluations to ensure the reliability and quality of the dataset. |
| Outcome: | The proposed framework improves accuracy and quality of the 2,000-synthesized datasets by integrating reasoning-dense filters, multi-agent collaboration, and human mathematicians’ evaluations. |
SARA: Salience-Aware Reinforced Adaptive Decoding for Large Language Models in Abstractive Summarization (2025.acl-long)
Copied to clipboard
| Challenge: | Existing decoding strategies neglect the explicit use of salient contextual information and rely on static hyperparameters to fix the balance between contextual and prior knowledge. |
| Approach: | They propose a salience-aware reinforced adaptive decoding (SARA) which incorporates salient contextual information and allows the model to determine reliance on source document's context, salient context, and model's prior knowledge based on pointwise mutual information. |
| Outcome: | The proposed model improves the quality and faithfulness of summaries across LLMs without modifying their weights. |
Simul-MuST-C: Simultaneous Multilingual Speech Translation Corpus Using Large Language Model (2024.emnlp-main)
Copied to clipboard
| Challenge: | Simultaneous speech translation (SiST) begins translating before the entire source input is received. |
| Approach: | They propose a dataset that rearranges sentences into segmented monotonic data for simultaneous speech translation using the Large Language Model. |
| Outcome: | The proposed dataset improves quality and latency in siST translations by rearranging sentences into segmented monotonic data. |
LADR: Locality-Aware Dynamic Rescue for Efficient Text-to-Image Generation with Diffusion Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for inference are expensive and lack spatial redundancy . Discrete Diffusion Language Models are a promising paradigm for multimodal generation . |
| Approach: | They propose a locality-aware dynamic rescue method that exploits spatial Markov property of images. |
| Outcome: | The proposed method achieves an approximate 4 speedup over baselines on four text-to-image generation benchmarks. |
Select High-quality Synthetic QA Pairs to Augment Training Data in MRC under the Reward Guidance of Generative Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches focus on downstream metrics to select QA pairs, which lack generalization across different datasets. |
| Approach: | They propose a general selection method that uses a large pre-trained language model as a reward model in a Reinforcement Learning framework for the training of the selection agent. |
| Outcome: | The proposed method improves performance on generative and extractive datasets. |
Balancing Diversity and Risk in LLM Sampling: How to Select Your Method and Parameter for Open-Ended Text Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies suggest that sampling-based decoding strategies can be used to optimize the output of Large Language Models (LLMs) . previous studies have shown that likelihood-maximization produces degenerate text which contains repetitive loops and incoherent context, especially in open-ended tasks. |
| Approach: | They propose to use a prefix tree to estimate the intrinsic capacity of a truncation sampling method by considering the trade-off between diversity and risk at each decoding step. |
| Outcome: | The proposed method is based on a prefix tree which preserves the context of a full sentence. |
Efficient Latent Semantic Clustering for Scaling Test-Time Computation of LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for scaling test-time computation rely on external models that introduce substantial computational overhead and fail to capture context-aware semantics. |
| Approach: | They propose a method that leverages the generator LLM’s internal hidden states for clustering, eliminating the need for external models. |
| Outcome: | The proposed method improves the computational efficiency of test-time scaling while maintaining or exceeding the performance of existing methods. |
Creating Grammar Teaching Material for Endangered Languages with Hybrid Grammar Induction (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for creating grammar lessons are labor-intensive and often fall to teachers who lack formal training in grammar. |
| Approach: | They propose a hybrid grammar-induction method that uses typological priors, Bayesian inference, constrained LLM reasoning and retrieval from sparse corpora to generate topic-specific grammar lessons. |
| Outcome: | The proposed method can produce coherent and useful lessons with better quality when modest explanatory evidence is available. |
PUER: Boosting Few-shot Positive-Unlabeled Entity Resolution with Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to entity resolution focus on supervised learning, but manual annotation is labor-intensive. |
| Approach: | They propose an end-to-end ER solution that leverages Large Language Models in PU learning setting to address low-resource entity resolution. |
| Outcome: | The proposed solution improves the performance of PUER on a positive-unlabeled learning environment. |
Query-driven Document-level Scientific Evidence Extraction from Biomedical Studies (2025.acl-long)
Copied to clipboard
Massimiliano Pronesti, Joao H Bettencourt-Silva, Paul Flanagan, Alessandra Pascale, Oisín Redmond, Anya Belz, Yufang Hou
| Challenge: | Systematic reviews are widely regarded as the gold standard in evidence-based medicine, heavily influencing medical decisions made by doctors, health authorities, and patients. |
| Approach: | They propose a retrieval-augmented generation framework to tackle the unique challenges of evidence extraction by leveraging forest plots from Cochrane systematic reviews. |
| Outcome: | The proposed framework outperforms existing methods by up to 10.3% in the F1 score on this task. |
Standard Quality Criteria Derived from Current NLP Evaluations for Guiding Evaluation Design and Grounding Comparability and AI Compliance Assessments (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations do not evaluate the same aspect of quality, resulting in unclear comparability and low repeatability. |
| Approach: | They propose to use a standard set of qualitycriterion names and definitions to establish comparability of existing evaluations. |
| Outcome: | The proposed taxonomy combines 114 quality criteria from 3 surveys of 933 evaluations in NLP and is used to establish comparability of existing evaluations and guide the design of new evaluations. |
CM-Align: Consistency-based Multilingual Alignment for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current large language models (LLMs) show a significant performance gap in alignment between English and other languages. |
| Approach: | They propose a consistency-based method to construct high-quality multilingual preference data for improving multilingual alignment. |
| Outcome: | The proposed method is based on three LLMs and three common tasks and shows that it performs better than current methods. |
Utilizing Longer Context than Speech Bubbles in Automated Manga Translation (2024.lrec-main)
Copied to clipboard
Hiroto Kaino, Soichiro Sugihara, Tomoyuki Kajiwara, Takashi Ninomiya, Joshua B. Tanner, Shonosuke Ishiwatari
| Challenge: | Existing methods to capture contextual information for manga machine translation are difficult to perform . unofficially translated pirated copies of manga are circulating overseas in large numbers . |
| Approach: | They propose two new ways to capture broader contextual information in manga machine translation . scene-based translation considers previous scene and broader context information . detailed analysis reveals the effect of zero-anaphora resolution in translation - highlighting the usefulness of longer contextual information if manga is translated in Japanese . |
| Outcome: | The proposed methods improve translation quality for manga (Japanese-style comics) the results show that the combined methods achieve the highest quality. |
Leveraging Outline-Optimized Generative Interactions and Critique for Self-Refining Outlines with Reinforcement Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Logic-RL is a framework that transforms critique-guided outline refinement into a learnable policy through reinforcement learning. |
| Approach: | They propose a framework that transforms critique-guided outline refinement into a learnable policy through reinforcement learning. |
| Outcome: | The proposed framework improves on FreshWiki and WikiOutline . it can be iteratively applied, with improved quality continuing through three refinement rounds before diminishing returns. |
CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges (2026.findings-acl)
Copied to clipboard
| Challenge: | Increasing saturation of web data limits further scaling of model intelligence. |
| Approach: | They propose a benchmark to evaluate machine creativity in code generation that combines combinatorial and exploratory creativity through reverse engineering and self-play. |
| Outcome: | The proposed benchmark targets combinatorial and exploratory creativity through reverse engineering and self-play. |
A Multilingual, Culture-First Approach to Addressing Misgendering in LLM Applications (2025.emnlp-main)
Copied to clipboard
| Challenge: | Misgendering is the act of referring to someone by using words that do not match their chosen identity. |
| Approach: | They propose to use a participatory-design approach to assess and mitigate misgendering across 42 languages and dialects using a human-in-the-loop approach. |
| Outcome: | The proposed guardrails reduce misgendering rates across all languages and dialects without loss of quality and without loss in quality. |
EMSDialog: Synthetic Multi-person Emergency Medical Service Dialogue Generation from Electronic Patient Care Reports via Multi-LLM Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing medical dialogue corpora are largely dyadic or lack multi-party workflow and annotations needed for this setting. |
| Approach: | They propose an ePCR-grounded, topic-flow-based multi-agent generation pipeline that iteratively plans, generates, and self-refines dialogues with rule-based factual and topic flow checks. |
| Outcome: | The proposed pipeline yields a dataset of 4,414 synthetic multi-speaker EMS conversations annotated with 43 diagnoses, speaker roles, and turn-level topics. |
Better LLM Reasoning via Dual-Play (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have made remarkable progress through Reinforcement Learning with Verifiable Rewards (RLVR) however, external supervision remains a bottleneck for tasks and domains for which supervised data are scarce or non-existent. |
| Approach: | They propose a novel dual-play framework that adversarially trains two models initialized from the same base model. |
| Outcome: | The proposed framework improves the math reasoning performance of large language models. |
Table-LLM-Specialist: Language Model Specialists for Tables using Iterative Fine-tuning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Language models such as GPT and Llama have shown remarkable ability on diverse natural language tasks, yet their performance on complex table tasks is suboptimal. |
| Approach: | They propose a generator-validator paradigm to iteratively generate-then-validate training data from language models to fine-tune stronger Table-Specialist models that can specialize in a given task, without using manually-labeled data. |
| Outcome: | The proposed model outperforms vanilla language models on diverse table tasks and can match or surpass GPT-4 level quality. |
Verified Critical Step Optimization for LLM Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Critical Step Optimization (CSO) focuses preference learning on verified critical steps where alternative actions demonstrably flip task outcomes from failure to success. |
| Approach: | They propose a method which focuses preference learning on verified critical steps where alternative actions demonstrably flip task outcomes from failure to success. |
| Outcome: | The proposed method outperforms the existing methods on GAIA-Text-103 and XBench-DeepSearch while requiring supervision at only 16% of trajectory steps. |
GUIDE: Towards Scalable Advising for Research Ideas (2026.acl-long)
Copied to clipboard
| Challenge: | Existing systems that provide detailed, constructive feedback on academic papers struggle with review fidelity. |
| Approach: | They explore factors that underlie the development of robust advising systems . large language models have shown remarkable progress in tasks from text generation to code synthesis . |
| Outcome: | The proposed model outperforms general-purpose language models in acceptance rates for self-ranked top-30% submissions to ICLR 2025. |
TripTide: A Benchmark for Adaptive Travel Planning under Disruptions (2026.findings-acl)
Copied to clipboard
Priyanshu Karmakar, Soumyabrata Chaudhuri, Shubhojit Mallick, Manish Gupta, Abhik Jana, Shreya Ghosh
| Challenge: | Recent work has shown the promise of Large Language Models (LLMs) for personalized, constraint-aware travel itinerary generation, but real-world travel often involves disruptions such as transit cancellations, weather-related closures, or overbooked attractions. |
| Approach: | They propose a benchmark to evaluate the ability of Large Language Models (LLMs) to revise travel itineraries under realistic disruptions. |
| Outcome: | The proposed benchmark evaluates the ability of Large Language Models (LLMs) to revise travel itineraries under real-world disruption scenarios. |
Beyond Evidence: Belief-Chain Conditioning for Persuasive Misinformation Debunking Explanation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to misinformation correction focus on relying on audience beliefs to generate factually accurate responses and to engage with users' mental states. |
| Approach: | They construct large language models with cognitive chains and use them to model their outputs on beliefs that engage with users' mental states. |
| Outcome: | The proposed model improves explanation quality for audiences with misinformation-aligned beliefs by incorporating believers’ chains into the model. |