Papers with precision
RAVEN: Robust Advertisement Video Violation Temporal Grounding via Reinforcement Reasoning (2025.acl-industry)
Copied to clipboard
| Challenge: | Existing methods for detecting ads video violations lack precise temporal grounding, noisy annotations, and limited generalization. |
| Approach: | They propose a framework that integrates curriculum reinforcement learning with large language models to enhance reasoning and cognitive capabilities for violation detection. |
| Outcome: | The proposed framework achieves superior performance in violation category accuracy and temporal interval localization. |
Rumor Detection on Twitter Using Multiloss Hierarchical BiLSTM with an Attenuation Factor (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing models to classify rumors have low precision and are time consuming. |
| Approach: | They propose a multiloss hierarchical biLSTM model with an attenuation factor that can extract deep information from limited quantities of text. |
| Outcome: | The proposed model can extract deep information from limited quantities of text. |
Enabling Search and Collaborative Assembly of Causal Interactions Extracted from Multilingual and Multi-domain Free Text (N19-4)
Copied to clipboard
George C. G. Barbosa, Zechy Wong, Gus Hahn-Powell, Dane Bell, Rebecca Sharp, Marco A. Valenzuela-Escárcega, Mihai Surdeanu
| Challenge: | a new searchable knowledge graph allows users to search for causal interactions in multiple languages . a recent study shows that search tools are shallow and do not support multilingual research . |
| Approach: | They propose a system that integrates causal interactions into a single searchable knowledge graph. |
| Outcome: | The proposed system extracts over 600 thousand causal statements from 120 thousand Portuguese publications with a precision of 62%. |
Learning as Abduction: Trainable Natural Logic Theorem Prover for Natural Language Inference (2020.starsem-1)
Copied to clipboard
| Challenge: | a logic-based approach to Natural Language Inference is becoming less and less common . a new method uses semantic relations to abduct sentences from data . |
| Approach: | They propose a method to reverse a theorem-proving procedure to abduct semantic relations from data. |
| Outcome: | The proposed method improves the performance of the theorem prover on the SICK dataset by 1.4% while maintaining high precision (>94%) |
BCTH: A Novel Text Hashing Approach via Bayesian Clustering (2020.aacl-main)
Copied to clipboard
| Challenge: | Similarity search is a promising strategy to find the most similar items for a given query item. |
| Approach: | They propose to utilize Bayesian Clustering for Text Hashing to map documents to binary codes by utilizing multiple Bayessian Clusters in parallel. |
| Outcome: | The proposed approach is competitive compared with baselines in the perspective of precision and training speed. |
GEAR: Augmenting Language Models with Generalizable and Efficient Tool Resolution (2024.eacl-long)
Copied to clipboard
| Challenge: | Recent work on Augmented Language Models (LLMs) over-rely on task-specific demonstrations that limits their generalizability and computational cost. |
| Approach: | They propose a query-tool grounding algorithm that is generalizable to various tasks . they delegate tool grounding and execution to small language models and LLMs . |
| Outcome: | The proposed algorithm outperforms baselines on 14 datasets and shows it can be generalized to different tasks. |
Fact Finder - Enhancing Domain Expertise of Large Language Models by Incorporating Knowledge Graphs (2026.eacl-demo)
Copied to clipboard
Daniel Steinigen, Roman Teucher, Timm Heine Ruland, Max Rudat, Nicolas Flores-Herr, Peter Fischer, Nikola Milosevic, Christopher Schymura, Angelo Ziletti
| Challenge: | Recent advances in Large Language Models have demonstrated their proficiency in answering natural language queries. |
| Approach: | They propose a system that augments Large Language Models with domain-specific knowledge graphs . they evaluate a medical KG and use a KG-based retrieval approach to enhance factual correctness . |
| Outcome: | The proposed system surpasses a standalone LLM in accuracy and completeness on a medical KG dataset. |
Ask-and-Verify: Span Candidate Generation and Verification for Attribute Value Extraction (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Existing reading comprehension models can over-generate attribute values which hinders precision. |
| Approach: | They propose a product attribute value extraction task that captures key factual information from product descriptions and a new end-to-end pipeline framework called Ask-and-Verify. |
| Outcome: | The proposed framework outperforms existing models by up to 3.1% F1 absolute improvement points while scaling to thousands of attributes. |
Personalized Dense Retrieval on Global Index for Voice-enabled Conversational Systems (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Constrained retrieval is limited to entities in recent user history, which offers low coverage of future requests. |
| Approach: | They propose a personalized entity retrieval system that is robust to phonetic noise and ambiguity but is not limited to a customized index. |
| Outcome: | The proposed system corrects multiple error modes and shows 91% improvement over baseline on the entity retrieval task. |
Cross-Task Knowledge Transfer for Query-Based Text Summarization (D19-58)
Copied to clipboard
| Challenge: | Existing methods for summarization data corpora are limited to extractive and abstractive summarizing. |
| Approach: | They propose to use machine reading comprehension (MRC) and query-based text summarization to produce extractive and abstractive summaries from pre-trained MRC and MT models. |
| Outcome: | The proposed model outperforms existing methods on CNN/Daily Mail and Debatepedia datasets and can be used as a baseline for future systems. |
PiKGL: Leveraging Pruned Knowledge Graphs for Explainable Stance Detection (2026.tacl-1)
Copied to clipboard
Bingbing Wang, Jingjie Lin, Zhixin Bai, Xintong Song, Qianlong Wang, Min Yang, Xi Zeng, Jing Li, Ruifeng Xu
| Challenge: | Experimental results demonstrate that a Pruned interpretable knowledge Graph Learning framework for explainable stance detection is state-of-the-art for social media stance prediction. |
| Approach: | They propose a Pruned interpretable knowledge Graph Learning framework for explainable stance detection that incorporates commonsense knowledge and prunes redundant information to ensure precision and minimize noise. |
| Outcome: | The proposed framework achieves state-of-the-art on three public datasets. |
AMR Alignment for Morphologically-rich and Pro-drop Languages (2022.acl-srw)
Copied to clipboard
| Challenge: | Existing AMR aligners for English are not well suited for many languages where many concepts appear from morphologically-semantic elements. |
| Approach: | They propose to use a tree traversal approach to align AMR concepts from morphemes in a Turkish language. |
| Outcome: | The proposed aligner outperforms the existing aligners for English and Portuguese in terms of precision, recall and F1 score. |
TALEN: Tool for Annotation of Low-resource ENtities (P18-4)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a task that requires a large amount of training data and annotators who do not speak the language are hard or impossible to find. |
| Approach: | They propose a web-based interface for named entity annotation in low-resource settings . TALEN includes in-place lexicon integration, TF-IDF token statistics, Internet search, and entity propagation . |
| Outcome: | The proposed interface performs better than a popular annotation tool and is more accurate and recall-rich than the current one. |
Diversity-aware Event Prediction based on a Conditional Variational Autoencoder with Reconstruction (D19-60)
Copied to clipboard
| Challenge: | Typical event sequences are important class of commonsense knowledge . previous work in event prediction uses sequence-to-sequence models . however, what can happen after a given event is usually diverse . |
| Approach: | They propose to incorporate a conditional variational autoencoder into seq2seq for its ability to represent diverse next events as a probabilistic distribution. |
| Outcome: | The proposed model outperforms deterministic models in terms of precision and recall . the proposed model is based on a conditional variational autoencoder . |
MWE as WSD: Solving Multiword Expression Identification with Word Sense Disambiguation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent approaches to word sense disambiguation use encodings of the sense gloss and context information to improve performance. |
| Approach: | They propose a poly-encoder architecture which uses the sense gloss to improve WSD performance. |
| Outcome: | The proposed approach outperforms the state-of-the-art in word sense disambiguation by 1.9 F1 points and on the PARSEME 1.1 English dataset. |
CONSTRUCTA: Automating Commercial Construction Schedules in Fabrication Facilities with Large Language Models (2025.naacl-industry)
Copied to clipboard
| Challenge: | Existing methods for automating construction scheduling are limited due to their domain knowledge and complexity. |
| Approach: | They propose a framework leveraging LLMs to optimize construction schedules in commercial construction. |
| Outcome: | The proposed framework improves missing value prediction, dependency analysis and planning performance compared to baseline methods. |
Learning When Not to Answer: a Ternary Reward Structure for Reinforcement Learning Based Question Answering (N19-2)
Copied to clipboard
| Challenge: | Existing methods for question answering over knowledge graphs use reinforcement learning to reason over a knowledge graph. |
| Approach: | They propose a new performance metric for question-answering agents that extends the binary reward structure to a ternary reward structure which rewards an agent for not answering a question rather than giving an incorrect answer. |
| Outcome: | The proposed method significantly improves the precision of answered questions while only not answering a limited number of correctly answered questions. |
A Multi-Task Learning Framework for Extracting Bacteria Biotope Information (D19-57)
Copied to clipboard
| Challenge: | Existing methods to extract information from unstructured text are slow or expensive to get. |
| Approach: | They propose a multi-task transfer multi-learning method for Bacteria Biotope rel+ner task . they use BERT and pre-train it using mask language models and next sentence prediction . |
| Outcome: | The proposed method achieves the best performance on all metrics including slot error rate, precision and recall in the Bacteria Biotope rel+ner subtask. |
Faceted Hierarchy: A New Graph Type to Organize Scientific Concepts and a Construction Method (D19-53)
Copied to clipboard
| Challenge: | faceted concept hierarchy is a structure of parent-child relationships . concepts are expected to be organized in a hierarchical structure for student learning . |
| Approach: | They propose a faceted concept hierarchy that aims to build facets from scientific literature. |
| Outcome: | The proposed hierarchy is more complete than "type-of" relations, and resolves conflicts by maintaining the acyclic structure of a hierarchy. |
Leveraging Product Catalog Patterns for Multilingual E-commerce Product Attribute Prediction (2025.emnlp-industry)
Copied to clipboard
| Challenge: | E-commerce stores increasingly use Large Language Models to improve catalog data quality . a critical challenge is accurately predicting missing structured attribute values . |
| Approach: | They propose a retrieval-augmented system that leverages existing product catalog entries to guide LLM predictions for missing attributes. |
| Outcome: | The proposed system improves catalog data quality by 34% and accuracy by 0.8% . the proposed model can predict missing attributes in multilingual product catalogs . |
Humor Recognition Using Deep Learning (N18-2)
Copied to clipboard
| Challenge: | Humor is an essential but most fascinating element in personal communication. |
| Approach: | They propose a convolutional neural network with extensive filter size and filter number to increase the depth of networks. |
| Outcome: | The proposed model outperforms existing models on accuracy, precision and recall . the proposed model can learn to distinguish between humorous and nonhumorous texts . |
Sparse, Dense, and Attentional Representations for Text Retrieval (2021.tacl-1)
Copied to clipboard
| Challenge: | Dual encoders perform retrieval by encoding documents and queries into dense low-dimensional vectors, scoring each document by its inner product with the query. |
| Approach: | They propose a dual-encoder-based neural model that combines the efficiency of dual encoders with expressiveness of more costly attentional architectures. |
| Outcome: | The proposed model outperforms strong alternatives in large-scale retrieval. |
ScopeIt: Scoping Task Relevant Sentences in Documents (2020.coling-industry)
Copied to clipboard
| Challenge: | a problem faced by conversational agents working with large documents is the frequent presence of information that is irrelevant to the agent. |
| Approach: | They propose a neural model for scoping relevant information from a large document . they show that the model performs better with emails than existing baselines . |
| Outcome: | The proposed model improves intent detection and entity extraction tasks without drop in recall. |
TreeRAG: Unleashing the Power of Hierarchical Storage for Enhanced Knowledge Retrieval in Long Documents (2025.findings-acl)
Copied to clipboard
| Challenge: | Traditional RAG frameworks struggle to retrieve all relevant knowledge points . a new approach to retrieve long documents is proposed to improve performance in NLP . |
| Approach: | They propose a tree-based approach to document knowledge retrieval that preserves hierarchical structure . treeRAG is a key technique for enhancing the text generation capabilities of Large Language Models . |
| Outcome: | The proposed approach improves recall quality and precision compared to existing methods and better performance to question-answering tasks. |
LLM-Powered Test Case Generation for Detecting Bugs in Plausible Programs (2025.acl-long)
Copied to clipboard
Kaibo Liu, Zhenpeng Chen, Yiyang Liu, Jie M. Zhang, Mark Harman, Yudong Han, Yun Ma, Yihong Dong, Ge Li, Gang Huang
| Challenge: | TrickCatcher generates test cases that pass existing tests yet contain bugs . a recent study found that tricky bugs are not detected by test suites . |
| Approach: | They propose an LLM-powered approach to generating test cases for uncovering bugs in plausible programs . they use a PUT and specification to generate program variants, an input generator and an Llm to construct test inputs . |
| Outcome: | The proposed approach achieves recall, precision, and F1 scores that are 1.80, 2.65, and 1.66 . trickCatcher generates program variants based on the program under test and its specification . |
Label Noise in Context (2020.acl-demos)
Copied to clipboard
| Challenge: | Label noise—incorrectly or ambiguously labeled training examples—can negatively impact model performance. |
| Approach: | They propose a noise-detection method that uses an example's neighborhood within the training set to reduce false positives and provide an explanation as to why the ex ample was flagged as noise. |
| Outcome: | The proposed method outperforms the state-of-the-art on precision and F0.5-score on short-text classification datasets. |
TMR: Evaluating NER Recall on Tough Mentions (2021.eacl-srw)
Copied to clipboard
| Challenge: | a NER evaluation tool is available via a repository. |
| Approach: | They propose to use Tough Mentions Recall to supplement traditional named entity recognition evaluation by examining recall on specific subsets of ”tough” mentions. |
| Outcome: | The proposed metrics enable differentiation between otherwise similar-scoring systems and identify patterns in performance that would go unnoticed from overall precision, recall, and F1. |
When and Why a Model Fails? A Human-in-the-loop Error Detection Framework for Sentiment Analysis (2021.naacl-industry)
Copied to clipboard
| Challenge: | Existing methods for sentiment analysis are difficult to assess for erroneous predictions that might exist prior to deployment. |
| Approach: | They propose a framework for error detection based on explainable features that can detect erroneous model predictions on unseen data with high precision. |
| Outcome: | The proposed framework detects erroneous model predictions on unseen data with high precision, given limited human-in-the-loop intervention, and can be deployed on unselected data with a high accuracy. |
ALiiCE: Evaluating Positional Fine-grained Citation Generation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing research on citation generation is limited to sentence-level statements . positional fine-grained citations can appear anywhere within sentences . |
| Approach: | They propose a framework that allows LLMs to generate citations from sentences . they use dependency tree-based methods to parse sentence-level claims into atomic claims . |
| Outcome: | The proposed framework evaluates citation quality using three metrics including positional fine-grained citation recall, precision, and coefficient of variation of citation positions. |
Discourse Structures Guided Fine-grained Propaganda Identification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Using teacher-predicted probabilities and knowledge distillation frameworks to identify propaganda content is important. |
| Approach: | They propose to integrate local and global discourse structures for propaganda discovery and construct two teacher models for identifying PDTB-style discourse relations between nearby sentences and common discourse roles of sentences in a news article respectively. |
| Outcome: | The proposed models improve accuracy and recall of propaganda content identification at sentence-level and token-level. |
Generate, Filter, and Rank: Grammaticality Classification for Production-Ready NLG Systems (N19-2)
Copied to clipboard
| Challenge: | Existing datasets for grammatical error correction don’t capture the distribution of errors that data-driven generators are likely to make. |
| Approach: | They propose a framework that allows candidates to be filtered and ranked to select the best response. |
| Outcome: | The proposed framework can be scaled with relatively low effort and achieve high precision with reasonable recall on a weather domain dataset. |
(Mis)alignment Between Stance Expressed in Social Media Data and Public Opinion Surveys (2021.emnlp-main)
Copied to clipboard
Kenneth Joseph, Sarah Shugars, Ryan Gallagher, Jon Green, Alexi Quintana Mathé, Zijian An, David Lazer
| Challenge: | Existing stance detection methods have been evaluated in comparison to the public opinion data they promise to replace. |
| Approach: | They propose to compare an individual's self-reported stance to the stance inferred from their social media data. |
| Outcome: | The proposed models are compared to a public opinion survey with 1,129 individuals across four salient targets. |
DuRE: Dual Contrastive Self Training for Semi-Supervised Relation Extraction (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing document-level relation extraction methods require manual training and labeled data to obtain supervised learning. |
| Approach: | They propose a document-level relation extraction framework that integrates RE and text generation as a dual process. |
| Outcome: | The proposed framework significantly boosts recall and F1 score with comparable precision on two document-level RE tasks against several strong baselines. |
SynKB: Semantic Search for Synthetic Procedures (2022.emnlp-demos)
Copied to clipboard
| Challenge: | SynKB is an open-source, automatically extracted knowledge base of chemical synthesis protocols. |
| Approach: | They propose to make SynKB available as an open-source tool for chemists . synKB supports more flexible queries about reaction conditions . |
| Outcome: | The proposed open-source tool has higher recall and high precision than proprietary chemistry databases. |
News Risk Alerting System (NRAS): A Data-Driven LLM Approach to Proactive Credit Risk Monitoring (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Credit risk monitoring is an essential process for financial institutions to evaluate the creditworthiness of borrowing entities. |
| Approach: | They propose a system which proactively alerts Credit Officers to credit-relevant news events . the system has been deployed for nearly three years and has an estimated precision of 77% . |
| Outcome: | The new system has alerted Credit Officers to over 2700 credit-relevant events with an estimated precision of 77%. |
Contextual ASR Error Handling with LLMs Augmentation for Goal-Oriented Conversational AI (2025.coling-industry)
Copied to clipboard
Yuya Asano, Sabit Hassan, Paras Sharma, Anthony B. Sicilia, Katherine Atwell, Diane Litman, Malihe Alikhani
| Challenge: | Existing ASR correction methods rely on prior user data or named entities . Existing methods based on prior data are not available for goal-oriented dialogues . |
| Approach: | They propose a method that integrates contextual information from the dialogue states of a goal-oriented conversational AI and its tasks into a large language model. |
| Outcome: | The proposed method improves recall and F1 of correction by 34% and 16% while maintaining precision and false positive rate. |
Reliable Evaluation Protocol for Low-Precision Retrieval (2026.acl-short)
Copied to clipboard
| Challenge: | Recent studies have shown that low-precision methods can improve performance, but they introduce high variability in the results based on tie resolution. |
| Approach: | They propose a retrieval evaluation protocol designed to reduce tie variation . high-precision scoring and tie-aware retrieval metrics are proposed to reduce this variability . |
| Outcome: | The proposed retrieval evaluation protocol reduces tie-induced instability and recovers expected scores and ranges on 12 retrieval datasets. |
AI for Climate Finance: Agentic Retrieval and Multi-Step Reasoning for Early Warning System Investments (2026.eacl-demo)
Copied to clipboard
Ario Saeid Vaghefi, Aymane Hachcham, Veronica Grasso, Nakiete Msemo, Chiara Colesanti Senni, Markus Leippold
| Challenge: | EWS financial flows are opaque and lack standardized labels, structures, and terminology for EWS-related spending. |
| Approach: | They propose an agent-based Retrieval-Augmented Generation system that uses hybrid retrieval and internal chain-of-thought reasoning to extract relevant financial data and classify EWS investments. |
| Outcome: | The proposed system outperforms four alternatives on multi-label classification and budget allocation on an annotated CREWS Fund corpus. |
NEST: Nested Evidence Survival for Retrieval (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing approaches to retrieval-augmented generation (RAG) rely on rigid heuristics or computational overhead. |
| Approach: | They propose a lightweight, training-free RAG framework that separates recall amplification from precision selection. |
| Outcome: | Evaluated on WebQuestions, HotpotQA and internalQA benchmarks, NEST outperforms strong adaptive RAG baselines. |
Training Language Models under Resource Constraints for Adversarial Advertisement Detection (2021.naacl-industry)
Copied to clipboard
| Challenge: | e-commerce and social media sites require content moderation to ensure ethical standards . a tiered moderation workflow with automated components complements human experts . |
| Approach: | They propose techniques for training text classification models under resource constraints . they use weak supervision, curriculum learning and multi-lingual training to fine-tune BERT . |
| Outcome: | The proposed techniques detect adversarial ads with a substantial gain over baseline . the authors show that the proposed methods can be applied to multiple languages . |
Building a Corpus from Handwritten Picture Postcards: Transcription, Annotation and Part-of-Speech Tagging (L18-1)
Copied to clipboard
| Challenge: | In this paper, we describe the processes and challenges of digitalisation, manual transcription, and manual annotation of over 11,000 postcards. |
| Approach: | They describe the processes and challenges of digitalisation, manual transcription, and manual annotation of over 11,000 postcards written in German and Swiss German. |
| Outcome: | The proposed system outperforms state-of-the-art taggers in the evaluation of the 'picture postcard corpus' containing over 11,000 handwritten postcards . |
A Differentiable Integer Linear Programming Solver for Explanation-Based Natural Language Inference (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing ILP frameworks are non-differentiable and cannot be integrated as part of a broader deep learning architecture. |
| Approach: | They propose a neuro-symbolic architecture for explanation-based NLI based on DBCS. |
| Outcome: | The proposed approach achieves superior performance when compared to existing solvers and black-box solver. |
The AI Committee: A Multi-Agent Framework for Automated Validation and Remediation of Web-Sourced Data (2026.eacl-demo)
Copied to clipboard
| Challenge: | largelanguage models (LLMs)-powered web agents can be useful for research in areas such as social science, public health, and economics. |
| Approach: | They propose a model-agnostic multi-agent system that auto-mates the process of validating and remediatingweb-sourced datasets. |
| Outcome: | The proposed system outperforms baseline approaches and achieves datacompleteness and precision up to 73.3%. |
LinkNav: Surfacing Interconnected Information in Scientific Articles (2026.acl-demo)
Copied to clipboard
| Challenge: | a non-linear reading order of academic literature is recognized by authors who make explicit connections between non-adjacent passages. |
| Approach: | They propose an enhanced reading experience which generates questions and searches for answer-bearing passages in academic papers to form intra-document connections when answers are found. |
| Outcome: | The proposed interface makes connections between related but non-adjacent passages even if the author did not make them explicit. |
Distributional Inclusion Vector Embedding for Unsupervised Hypernymy Detection (N18-1)
Copied to clipboard
| Challenge: | Existing unsupervised methods for learning hypernyms from unlabeled text are not scaled to large vocabularies or yield unacceptably poor accuracy. |
| Approach: | They propose an unsupervised method of hypernym discovery using word contexts . they use word2vec to embed word context distributions without supervision . |
| Outcome: | The proposed method provides double the precision and highest average performance on 11 datasets. |
Learning from Litigation: Graphs for Retrieval and Reasoning in eDiscovery (2025.acl-industry)
Copied to clipboard
| Challenge: | Electronic Discovery (eDiscovery) requires identifying relevant documents from vast collections for legal production requests. |
| Approach: | They propose a system that integrates knowledge graphs for enhanced document ranking and classification, augmented by LLM-driven reasoning. |
| Outcome: | The proposed system outperforms baselines in F1-score, precision, and recall across balanced and imbalanced datasets. |
Automated Writing Support Using Deep Linguistic Parsers (2020.lrec-1)
Copied to clipboard
Luís Morgado da Costa, Roger V P Winder, Shu Yun Li, Benedict Christopher Lin Tzer Liang, Joseph Mackinnon, Francis Bond
| Challenge: | Automated Grammar Error Detection (GED) and Grammar Erreor Correction (GEC) are tasks that have attracted some attention within the NLP community. |
| Approach: | They propose a web-based system that integrates English Grammatical Error Detection (GED) and course-specific stylistic guidelines to automatically review and provide feedback on student assignments. |
| Outcome: | The system integrates both general NLP methods and high precision parsers to check student assignments before they are submitted for grading. |
Prototype-Representations for Training Data Filtering in Weakly-Supervised Information Extraction (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Weak supervision and data programming are powerful tools to support information extraction models. |
| Approach: | They propose a prototype-based method to denoise weakly supervised training data . they use a model to model the correct contexts for a given target value . |
| Outcome: | The proposed method achieves 9% accuracy gain in attribute value extraction in e-commerce websites. |
Multimodal Generation of Radiology Reports using Knowledge-Grounded Extraction of Entities and Relations (2022.aacl-main)
Copied to clipboard
Francesco Dalla Serra, William Clackett, Hamish MacKinnon, Chaoyang Wang, Fani Deligianni, Jeff Dalton, Alison Q. O’Neil
| Challenge: | Existing approaches to generate text radiology reports are prone to errors and poor clinical accuracy. |
| Approach: | They propose a two-step pipeline that subdivides the problem into factual triple extraction followed by free-text report generation. |
| Outcome: | The proposed pipeline shows that the generated reports exhibit realistic style but lack clinical accuracy. |
Constructing A Dataset of Support and Attack Relations in Legal Arguments in Court Judgements using Linguistic Rules (2022.lrec-1)
Copied to clipboard
| Challenge: | Argumentation mining is a growing area of research with several interesting practical applications. |
| Approach: | They propose three sets of rules based on linguistic knowledge and distant supervision to identify such relations from Indian Supreme Court judgments. |
| Outcome: | The proposed rules are based on linguistic knowledge and distant supervision and use the source of the argument to build a dataset of Support and Attack relations between sentences in a court judgement with reasonable accuracy. |
The Impact of Language on Arithmetic Proficiency: A Multilingual Investigation with Cross-Agent Checking Computation (2024.naacl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) have garnered significant attention over the past year . previous studies have evaluated LLMs' performance in solving math word problems, but there is little discussion on whether they comprehend the operations they generate. |
| Approach: | They challenge the notion that arithmetic is language-independent and compare models with cross-agent collaborations to find significant limitations in their performance. |
| Outcome: | The proposed model outperforms collaborative approaches in basic arithmetic tasks. |
DIALECTIC: A Multi-Agent System for Startup Evaluation (2026.eacl-industry)
Copied to clipboard
| Challenge: | Venture capital (VC) investors face a large number of investment opportunities but only invest in few of them. |
| Approach: | They propose an LLM-based system that gathers factual knowledge about a startup and organizes it into a question tree. |
| Outcome: | The proposed system matches the precision of human VCs in predicting startup success. |
Long-Context Long-Form Question Answering for Legal Domain (2026.eacl-industry)
Copied to clipboard
| Challenge: | Legal documents have complex document layouts involving multiple nested sections and lengthy footnotes that make question answering challenging. |
| Approach: | They propose a question answering system that parses document layouts while isolating sections and footnotes and linking them appropriately. |
| Outcome: | The proposed system can parse complex document layouts while isolating sections and footnotes and linking them appropriately. |
Negation Detection in Dutch Spoken Human-Computer Conversations (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing negation detection methods in English are not available. |
| Approach: | They propose to annotate a Dutch dialogue corpus with negation cues and their scopes. |
| Outcome: | The proposed method can detect negation cues and scope in Dutch dialogues with high precision and recall. |
Semantic Outlier Removal with Embedding Models and LLMs (2025.acl-industry)
Copied to clipboard
Eren Akbiyik, João F. M. De Almeida, Rik Melis, Ritu Sriram, Viviana Petrescu, Vilhjálmur Vilhjálmsson
| Challenge: | Modern text processing pipelines require robust methods to remove extraneous content while preserving a document’s core message. |
| Approach: | They propose a method that leverages multilingual sentence embeddings and approximate nearest-neighbor search to identify and excise unwanted text segments. |
| Outcome: | Experiments on HTML datasets show that SORE outperforms structural methods and yields high precision in diverse scenarios. |
Pretraining and Finetuning Language Models on Geospatial Networks for Accurate Address Matching (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing approaches to address matching and building authoritative address catalogues are limited in data quality and require labeling effort to develop accurate models. |
| Approach: | They propose to view addresses as an address graph and curate inputs by placing geospatially linked addresses in the same context. |
| Outcome: | The proposed framework improves address matching and fine-tuning language models. |
AutoKB: Automated Creation of Structured Knowledge Bases for Domain-Specific Support (2025.naacl-industry)
Copied to clipboard
| Challenge: | Effective customer support requires domain-specific solutions tailored to users’ issues. |
| Approach: | They propose an automated pipeline for building a domain-specific KB with a hierarchical tree structure that maps user issues to precise and domain-compliant solutions. |
| Outcome: | Experiments in troubleshooting and medical domains show that the proposed pipeline outperforms LLMs and unstructured knowledge bases and is 75 times more cost-effective than manual methods. |
A Hybrid Supervised-LLM Pipeline for Actionable Suggestion Mining in Unstructured Customer Reviews (2026.eacl-industry)
Copied to clipboard
| Challenge: | Existing approaches to extract actionable suggestions from customer reviews are often mixed-intent, unstructured text. |
| Approach: | They propose a hybrid pipeline that uses a RoBERTa classifier and a precision–recall surrogate to extract actionable suggestions from customer reviews. |
| Outcome: | The proposed pipeline outperforms prompt-only, rule-based, and classifier-only baselines in extraction accuracy and cluster coherence. |
Aligning Large Language Models via Fine-grained Supervision (2024.acl-short)
Copied to clipboard
| Challenge: | Pre-trained large-scale language models often generate biased or toxic text, misaligning with human intentions. |
| Approach: | They propose to use human feedback to improve LLM alignment by fine-grained token supervision . they ask annotators to edit less preferred responses to make them more favorable . |
| Outcome: | The proposed method improves LLM alignment by up to 5.1% in terms of win rate compared with the traditional model. |
Predicting Customer Satisfaction with Soft Labels for Ordinal Classification (2023.acl-industry)
Copied to clipboard
| Challenge: | a typical call center only responds to 8% of customers with a customer satisfaction survey . a predictive algorithm that infers CSAT on the 1-5 scale is needed to minimize this data sparsity and response bias. |
| Approach: | They propose an algorithm that infers CSAT on 1-5 scale on inbound calls to the call center . they reframe the problem into a binary class and map it back to five classes . |
| Outcome: | The proposed model is able to support keycustomer workflows with high accuracy overmillions of calls a month. |
Auto prompting without training labels: An LLM cascade for product quality assessment in e-commerce catalogs (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Our system generates and refines prompts for evaluating attribute quality across tens of thousands of product category–attribute pairs. |
| Approach: | They propose a free cascade for auto-prompting Large Language Models (LLMs) that generates and refines prompts for evaluating attribute quality across tens of thousands of product category–attribute pairs. |
| Outcome: | The proposed system improves precision and recall by 8–10% over chain-of-thought prompting while reducing domain expert effort from 5.1 hours to 3 minutes per attribute. |
From Spelling to Grammar: A New Framework for Chinese Grammatical Error Correction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Chinese Grammatical Error Correction (CGEC) aims to generate correct sentences from erroneous sequences. |
| Approach: | They propose a zero-shot approach for spelling error correction that is simple but effective . they propose auxiliary task to predict POS sequence of target sentence . |
| Outcome: | The proposed framework achieves 42.11 F-0.5 on the English GEC dataset outperforms the previous state-of-the-art by a wide margin of 1.30 points. |
SAKI-RAG: Mitigating Context Fragmentation in Long-Document RAG via Sentence-level Attention Knowledge Integration (2025.emnlp-main)
Copied to clipboard
| Challenge: | Traditional Retrieval-Augmented Generation (RAG) frameworks segment documents into larger chunks to preserve contextual coherence . however, such chunking methods lead to fragmented contexts, isolated chunk semantics, and broken inter-chunk relationships . |
| Approach: | They propose a framework that maintains granular chunks while recovering their intrinsic semantic connections. |
| Outcome: | The proposed framework achieves better recall and precision compared to other RAG frameworks in long-document retrieval scenarios. |
CompactIE: Compact Facts in Open Information Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Despite advances in open information extraction, many systems focus on covering more information over compactness of constituents. |
| Approach: | They propose a neural OpenIE system that produces compact extractions with overlapping constituents by using a pipelined approach. |
| Outcome: | The proposed system produces 1.5x-2x more compact extractions than previous systems, with high precision, establishing a new state-of-the-art in OpenIE. |
Korean L2 Vocabulary Prediction: Can a Large Annotated Corpus be Used to Train Better Models for Predicting Unknown Words? (L18-1)
Copied to clipboard
| Challenge: | a recent study suggests that a classifier trained on unknown words may yield better results for L2 learners. |
| Approach: | They propose to use a supervised learning classifier to predict word complexity in Korean . they propose to train models on annotated corpus of unknown words with 71 % precision . |
| Outcome: | The proposed model recalls 80 % of unknown words with 71 % precision. |
AutoBool: Reinforcement-Learned LLM for Effective Automatic Systematic Reviews Boolean Query Generation (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing approaches to generate Boolean queries for systematic reviews are limited by the lack of ground-truth best Boolesan queries. |
| Approach: | They propose a reinforcement learning framework that trains large language models to generate effective Boolean queries for medical systematic reviews. |
| Outcome: | The proposed framework outperforms zero-shot/few-shot prompting on 65 588 topics . it also matches or exceeds the effectiveness of larger GPT-based models using smaller backbones . |
Generalizing Unmasking for Short Texts (N19-1)
Copied to clipboard
| Challenge: | Authorship verification is the problem of inferring whether two texts were written by the same author. |
| Approach: | They propose a generalized unmasking approach which allows for authorship verification of short texts with high precision at an adjustable recall tradeoff. |
| Outcome: | The proposed approach achieves accuracies of 75–80% while allowing for easy adjustment to forensic scenarios that require higher levels of confidence. |
CoPHE: A Count-Preserving Hierarchical Evaluation Metric in Large-Scale Multi-Label Text Classification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Large-Scale Multi-Label Text Classification (LMTC) tasks with hierarchical label spaces include automatic assignment of ICD-9 codes to discharge summaries. |
| Approach: | They propose a set of metrics for hierarchical evaluation using the depth of the ontology to evaluate the predictions of neural LMTC models. |
| Outcome: | The proposed metrics compare with previous evaluations on prior art models for ICD-9 coding in MIMIC-III and propose further avenues of research involving the proposed representation. |
Learning Faithful Representations of Causal Graphs (2021.acl-long)
Copied to clipboard
| Challenge: | Existing text embeddings that predict direct causal links fail to capture other indirect causal links, leading to spurious correlations in downstream tasks. |
| Approach: | They define faithfulness property of contextual embeddings to capture geometric distance-based properties of directed acyclic causal graphs. |
| Outcome: | The embeddings are 31.3% more faithful to human validated graphs with 800K and 200K causal links and achieve better Precision-Recall AUC in a link prediction fine-tuning task. |
Improving Data Annotation for Low-Resource Relation Extraction with Logical Rule-Augmented Collaborative Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Low-resource relation extraction aims to identify semantic relationships using scarce labeled data. |
| Approach: | They propose a framework that iteratively integrates high-confidence predictions of rule-enhanced relation extractors with varying scales to obtain reliable pseudo annotations from massive unlabeled samples without human supervision. |
| Outcome: | The proposed framework achieves state-of-the-art on benchmark datasets in few-shot scenarios. |
Bottom-Up Synthesis of Knowledge-Grounded Task-Oriented Dialogues with Iteratively Self-Refined Prompts (2025.naacl-short)
Copied to clipboard
| Challenge: | Training conversational question-answering systems requires in-domain data, which is often scarce in practice. |
| Approach: | They propose a bottom-up approach where QA pairs are generated first and combined into a coherent dialogue. |
| Outcome: | The proposed approach produces more realistic and higher-quality dialogues compared to top-down methods. |
PDFMathTranslate: Scientific Document Translation Preserving Layouts (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Prior efforts in translating scientific documents overlooked layouts . PDFMathTranslate is open-source with more than 222k downloads - a record for the first time ever. |
| Approach: | They propose PDFMathTranslate, the world's first open-source software for translating scientific documents while preserving layouts. |
| Outcome: | The work is open-sourced at https://github.com/byaidu/pdfmathtranslate with more than 222k downloads. |
Developing a Reliable, Fast, General-Purpose Hallucination Detection and Mitigation Service (2025.naacl-industry)
Copied to clipboard
| Challenge: | Hallucination is a problem in large language models that produce incorrect output . authors propose a reliable and high-speed production system to detect and rectify hallucinations . |
| Approach: | They propose a high-speed production system that detects hallucinations in LLMs . they propose NER, natural language inference, span-based detection and a rewriting mechanism . |
| Outcome: | The proposed system detects a wide range of hallucinations in LLM responses. |
Using Crowd Agreement for Wordnet Localization (L18-1)
Copied to clipboard
| Challenge: | Lexical-semantic resources like WordNet are a fundamental resource for many NLP and semantic applications. |
| Approach: | They propose a crowdsourcing workflow that consists of synset localization and validation . they use inter-rater agreement metrics to estimate the precision of the results . |
| Outcome: | The proposed method is cost-effective and provides a good trade-off between quality and speed of progress. |
DIAGRAMS : A Review Framework for Reasoning-Level Attribution in Diagram QA (2026.acl-demo)
Copied to clipboard
Anirudh Iyengar Kaniyar Narayana Iyengar, Tampu Ravi Kumar, Manan Suri, Raviteja Bommireddy, Dinesh Manocha, Puneet Mathur, Vivek Gupta
| Challenge: | Diagram question answering (Diagram QA) requires reasoning-level attribution that links each question-answer pair to all visual regions needed to derive the answer. |
| Approach: | They propose a diagram question-answer review framework that decouples interface logic from dataset-specific JSON structures through an internal meta-schema and dataset adapters. |
| Outcome: | The proposed framework achieves 85.39% precision and 75.30% recall across six diagram QA datasets. |
Supervised and Nonlinear Alignment of Two Embedding Spaces for Dictionary Induction in Low Resourced Languages (D19-1)
Copied to clipboard
| Challenge: | Existing methods for mapping monolingual word embeddings into another are based on anchor points and unsupervised methods are more adversarial. |
| Approach: | They propose a noise-tolerant piecewise linear technique to learn a non-linear mapping between two monolingual word embedding vector spaces. |
| Outcome: | The proposed method outperforms the state-of-the-art in lower resourced settings with an average of 3.7% improvement of precision @10 across 14 mostly low resourced languages. |
Adjusting the Precision-Recall Trade-Off with Align-and-Predict Decoding for Grammatical Error Correction (2022.acl-short)
Copied to clipboard
| Challenge: | Modern writing assistance applications always contain a Grammatical Error Correction (GEC) model to correct errors in user-entered sentences. |
| Approach: | They propose a simple yet effective approach to Align-and-Predict Decoding for most popular sequence-to-sequence models to offer more flexibility for the precision-recall trade-off. |
| Outcome: | The proposed model can be used in both English and Chinese GEC models and achieve state-of-the-art results. |
Automated Prediction of Examinee Proficiency from Short-Answer Questions (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to predicting examinee proficiency from short-answer questions (SAQs) use of labeled data to train on is difficult, and requires expensive expert-rated data. |
| Approach: | They propose a method to predict examinee proficiency from short-answer questions . previous approaches train on manually labeled data to predict human-ratings assigned to SAQs . |
| Outcome: | The proposed model examines examinee proficiency directly and does not require manual training on labeled data. |
MEBench: Benchmarking Large Language Models for Cross-Document Multi-Entity Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) and Retrieval-augmented Generation (RAG) systems show promise, but their performance on cross-document MEQA remains underexplored due to the lack of tailored benchmarks. |
| Approach: | They propose a scalable multi-document, multi-entity benchmark to evaluate LLMs' capacity to retrieve, consolidate, and reason over scattered and dense information. |
| Outcome: | The proposed benchmarks show that even advanced models achieve only 59% accuracy on MEBench. |
Relational World Knowledge Representation in Contextual Language Models: A Review (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge bases are organized according to manual schemas that limit their expressiveness and require significant human engineering and maintenance. |
| Approach: | They propose to organize knowledge representation strategies in LMs by the level of KB supervision provided . they propose to highlight notable models, evaluation tasks, and findings . |
| Outcome: | The proposed model can internalize and express relational knowledge in more flexible forms. |
Point Precisely: Towards Ensuring the Precision of Data in Generated Texts Using Delayed Copy Mechanism (C18-1)
Copied to clipboard
| Challenge: | Recent neural generation systems have shown significant progress on data-to-text generation tasks. |
| Approach: | They propose a two-stage approach with a delayed copy mechanism to improve the precision of data records in the generated texts. |
| Outcome: | The proposed approach improves the accuracy of the generated texts on a RotoWire dataset. |
ASTRA: Automatic Schema Matching using Machine Translation (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Currently, eCommerce platforms use schema matching to structure product information from disparate sources. |
| Approach: | They propose to model the schema matching problem as a neural machine translation task . they propose to use open-source seq2seq models fine-tuned on product attribute mappings to build a framework . |
| Outcome: | The proposed model achieves a significant performance boost (15% precision and 7% recall uplift) it can support new attributes with precision 95% using only five labeled samples per attribute. |
A Human Subject Study of Named Entity Recognition in Conversational Music Recommendation Queries (2023.eacl-main)
Copied to clipboard
| Challenge: | NER is a complex task that requires a high degree of precision and a higher level of recall. |
| Approach: | They evaluated the human NER linguistic behaviour on a noisy corpus of conversational music recommendation queries with many irregular and novel named entities. |
| Outcome: | The results show that human NER was hard to perform under a strict evaluation schema and that the model had higher recall because of entity exposure. |
TOBUGraph: Knowledge Graph-Based Retrieval for Enhanced LLM Performance Beyond RAG (2025.emnlp-industry)
Copied to clipboard
Savini Kashmira, Jayanaka L. Dantanarayana, Joshua Brodsky, Ashish Mahendra, Yiping Kang, Krisztian Flautner, Lingjia Tang, Jason Mars
| Challenge: | Retrieval-Augmented Generation (RAG) relies on query-chunk text-to-text similarity in the embedding space for retrieval, can fail to capture deeper semantic relationships across chunks, is highly sensitive to chunking strategies, and is prone to hallucinations. |
| Approach: | They propose a graph-based retrieval framework that first constructs the knowledge graph from unstructured data dynamically and automatically. |
| Outcome: | The proposed framework outperforms multiple RAG implementations in both precision and recall, significantly enhancing user experience through improved retrieval accuracy. |
End-to-End Beam Retrieval for Multi-Hop Question Answering (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing beam retrieval frameworks for multi-hop question answering were customized for two-hop questions and were poorly supervised. |
| Approach: | They propose an end-to-end beam retrieval framework for multi-hop question answering . they combine an encoder and two classification heads to optimize the retrieval process . |
| Outcome: | The proposed framework improves on MuSiQue-Ans and surpasses all previous retrievers on HotpotQA and achieves 99.9% precision on 2WikiMultiHopQA. |
I-SEE: An Instruction-tuned, SOP-Enhanced Quality Evaluator for Product Content (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing approaches to content evaluation treat information uniformly without prioritizing based on customer relevance. |
| Approach: | They propose a framework that combines domain expertise with a single instruction to improve content. |
| Outcome: | a new framework outperforms existing models in detecting inconsistencies across 20 product categories and 150 product specific features. |
Fin-ExBERT: User Intent based Text Extraction in Financial Context using Graph-Augmented BERT and trainable Plugin (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Financial dialogue transcripts pose a unique challenge for sentence-level information extraction due to their informal structure, domain-specific vocabulary, and variable intent density. |
| Approach: | They propose a framework for extracting user intent–relevant sentences from financial service calls. |
| Outcome: | The proposed framework shows strong precision and F1 performance on real-world transcripts . financial transcripts are a challenge due to their informal structure and domain-specific vocabulary . |
Generalized chart constraints for efficient PCFG and TAG parsing (P18-2)
Copied to clipboard
| Challenge: | Existing pruning techniques limit chart constraints to PCFGs and cannot be applied to more expressive grammars. |
| Approach: | They propose to apply chart constraints to more expressive grammars and a neural tagger which predicts chart constraints at very high precision. |
| Outcome: | The proposed technique accelerates both PCFG and TAG parsing by two orders of magnitude while improving accuracy. |
Increasing Coverage and Precision of Textual Information in Multilingual Knowledge Graphs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to generate knowledge graphs are unable to handle non-English textual information. |
| Approach: | They propose a task of automatic Knowledge Graph Completion to bridge the gap between English and non-English textual information. |
| Outcome: | The proposed method bridges the gap between the quantity and quality of textual information between English and non-English languages. |
Effective Performance Measurement: Challenges and Opportunities in KPI Extraction from Earnings Calls (2026.acl-industry)
Copied to clipboard
Rasmus T. Aavang, Rasmus Tjalk-Bøggild, Alexandre Iolov, Giovanni Rizzi, Mike Zhang, Johannes Bjerva
| Challenge: | Earnings calls are a key source of financial information about public companies. extracting information from earnings calls is difficult. |
| Approach: | They propose to use LLMs to perform open-ended extraction from unstructured call transcripts to provide a baseline for this valuable domain through the consistent tracking of emergent KPIs. |
| Outcome: | The proposed method provides a baseline for this valuable domain through the consistent tracking of emergent KPIs. |
A Submodular Feature-Aware Framework for Label Subset Selection in Extreme Classification Problems (N19-1)
Copied to clipboard
| Challenge: | Experimental results show that extreme multi-label learning improves label prediction quality by 3% to 5% in three of the 5 tasks and is competitive in the others. |
| Approach: | They propose a submodular maximization framework with linear cost to find informative labels which are most relevant to other labels yet least redundant with each other. |
| Outcome: | The proposed model improves label prediction quality by 3% to 5% in three of the 5 tasks and is competitive in the others. |
EXPLAIN: Enhancing Retrieval-Augmented Generation with Entity Summary (2025.acl-industry)
Copied to clipboard
Yaozhen Liang, Xiao Liu, Jiajun Yu, Zhouhua Fang, Qunsheng Zou, Linghan Zheng, Yong Li, Zhiwei Liu, Haishuai Wang
| Challenge: | Existing document question answering methods reduce inference costs and input tokens. |
| Approach: | They propose a retrieval-augmented generation method that automatically extracts useful entities and generates summaries from documents. |
| Outcome: | The proposed method surpasses baseline retrieval-augmented generation (RAG) and long-context question answering (LC) methods achieve higher accuracy by processing entire documents, but at the cost of increased computational Corresponding authors. |
Transc&Anno: A Graphical Tool for the Transcription and On-the-Fly Annotation of Handwritten Documents (L18-1)
Copied to clipboard
| Challenge: | Transc&Anno is a web-based collaboration tool for linguists to facilitate the transcription of text images and their shallow on-the-fly annotation. |
| Approach: | They propose a web-based collaboration tool that allows the transcription of text images and their shallow on-the-fly annotation. |
| Outcome: | The Transc&Anno tool can be used for any type of corpora requiring transcription and shallow on-the-fly annotation resulting in inline XML. |
Correction of OCR Word Segmentation Errors in Articles from the ACL Collection through Neural Machine Translation Methods (L18-1)
Copied to clipboard
| Challenge: | Optical Character Recognition (OCR) can produce a range of errors depending on the quality of the original document. |
| Approach: | They applied a sequence-to-sequence machine translation system to correct word-single-word OCR errors in scientific texts from the ACL collection with an estimated precision and recall above 0.95 on test data. |
| Outcome: | The proposed system corrects word-segmentation OCR errors with an estimated precision and recall above 0.95 on test data. |
Factual State Discovery Benchmark: Evaluating Fact Elicitation in Polish Tax Law (2026.acl-srw)
Copied to clipboard
Mateusz Bystroński, Kamil Tagowski, Denis Janiak, Julia Farganus, Lukasz Augustyniak, Monika Kajdanowicz, Tomasz Jan Kajdanowicz
| Challenge: | FSDBench is a benchmark for eliciting all relevant facts through dialogue . missing facts may lead the authority to apply the wrong provision or issue a ruling that is inapplicable to the actual situation. |
| Approach: | They propose a method to systematically elicit facts through dialogue from a simulated taxpayer . they use 500 narratives from official Polish tax interpretations to test their models . |
| Outcome: | The proposed model recovers only 77% of facts on easy and hard samples and under 49% on hard samples after 50 turns. |
On the Practical Computational Power of Finite Precision RNNs for Language Recognition (P18-2)
Copied to clipboard
| Challenge: | Recurrent Neural Networks (RNNs) are famously known to be Turing complete, but this relies on infinite precision in the states and unbounded computation time. |
| Approach: | They propose to use LSTM and Elman-RNN with ReLU activation to study RNNs . they show that LS and ReLU-RNns can easily implement counting behavior . |
| Outcome: | The LSTM and the Elman-RNN with ReLU activation are stronger than the RNN with squashing activation and the GRU. |
PRISM: A New Lens for Improved Color Understanding (2024.emnlp-industry)
Copied to clipboard
| Challenge: | PRISM is a visual representation learner that can grasp the nuances of precise colors without compromising CLIP’s performance on established benchmarks. |
| Approach: | They propose a method that extends CLIP's ability to grasp the nuances of precise colors by utilizing a curated dataset of 100 image-text pairs that can be effortlessly repurposed for fine-tuning. |
| Outcome: | The proposed method improves CLIP's ability to grasp the nuances of precise colors without compromising CLIP’s performance on established benchmarks. |
Self-Regulated Sample Diversity in Large Language Models (2024.findings-naacl)
Copied to clipboard
Mingyue Liu, Jonathan Frawley, Sarah Wyer, Hubert P. H. Shum, Sara Uckelman, Sue Black, Chris Willcocks
| Challenge: | Existing methods that require expensive setups or maintain static values during inference are inflexible and require expensive training. |
| Approach: | They propose a self-regulating approach that adjusts sample diversity parameters dynamically based on the input prompt. |
| Outcome: | The proposed method significantly improves the quality of responses generically without model retraining or fine-tuning. |
Visual Referring Expression Recognition: What Do Systems Actually Learn? (N18-2)
Copied to clipboard
| Challenge: | Existing systems for referring expression recognition ignore linguistic structure, instead relying on shallow correlations introduced by unintended biases in the data selection and annotation process. |
| Approach: | They propose to use a system trained on the input image without the input referring expression to achieve a precision of 71.2% in top-2 predictions. |
| Outcome: | The proposed model can achieve 71.2% accuracy on the input image without the input referring expression and 84.2% on the object category given the input. |
Cross-lingual Knowledge Projection Using Machine Translation and Target-side Knowledge Base Completion (C18-1)
Copied to clipboard
| Challenge: | Existing efforts to build commonsense knowledge bases are expensive and lack quantity and quality between languages. |
| Approach: | They propose to project English commonsense knowledge into Japanese and Chinese with high precision. |
| Outcome: | The proposed method achieves top-10 accuracy on the crowdsourced English–Japanese benchmark and 18,747 facts of accurate Japanese commonsense within a very short period. |
Unveiling Decision-Making in LLMs for Text Classification : Extraction of influential and interpretable concepts with Sparse Autoencoders (2026.findings-eacl)
Copied to clipboard
| Challenge: | Concept-based explanations for large language models are not well understood in text classification. |
| Approach: | They propose a model with a specialized classifier head and activation rate sparsity loss for sentence classification . they compare it to existing models with HI-Concept and ConceptShap . |
| Outcome: | The proposed model improves both the causality and interpretability of the extracted features. |
Coalescing Global and Local Information for Procedural Text Understanding (2022.coling-1)
Copied to clipboard
| Challenge: | Existing models for procedural text understanding have low precision or low recall . et al., 2012, pp. 106-106. |
| Approach: | They propose a model that builds entity- and timestep-aware input representations . they extend the model with additional output layers and integrate it into a story reasoning framework . |
| Outcome: | The proposed model achieves state-of-the-art on a popular procedural text understanding dataset and on 'story reasoning benchmark' it integrates the model with additional output layers and improves on the previous models. |
Unlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to compress KV cache compromise precision or require extra data for calibration, limiting their practicality in LLM deployment. |
| Approach: | They propose a low-bit quantization technique based on tensor decomposition to effectively compress KV cache. |
| Outcome: | The proposed method reduces memory footprint and performance by 75% . it is compared with existing methods that compromise precision or require extra data for calibration . |
Beyond Pointwise Scores: Decomposed Criteria-Based Evaluation of LLM Responses (2025.emnlp-industry)
Copied to clipboard
| Challenge: | DeCE is model-agnostic and domain-general, requiring no predefined taxonomies or handcrafted rubrics. |
| Approach: | They propose a decomposed LLM evaluation framework that separates accuracy and recall from accuracy and relevance. |
| Outcome: | The proposed framework achieves stronger correlation with expert judgments than traditional metrics and pointwise LLM scoring. |
GraphRel: Modeling Text as Relational Graphs for Joint Entity and Relation Extraction (P19-1)
Copied to clipboard
| Challenge: | GraphRel is an end-to-end relation extraction model that uses graph convolutional networks to learn named entities and relations. |
| Approach: | They propose a graph-based relation extraction model which uses graph convolutional networks to jointly learn named entities and relations. |
| Outcome: | The proposed model outperforms previous models on two public datasets: NYT and WebNLG. |
Crawling The Internal Knowledge-Base of Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for representing factual knowledge in a language model are insufficient. |
| Approach: | They propose a procedure for “crawling” the internal knowledge-base of a language model by expanding a knowledge-graph around it. |
| Outcome: | The proposed method yields high precision graphs (82-92%) while emitting a reasonable number of facts per entity. |
LLMDet: A Third Party Large Language Models Generated Text Detection Tool (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing detection tools rely on access to LLMs and can only distinguish between machine-generated and human-authored text. |
| Approach: | They propose a model-specific, secure, efficient, and extendable detection tool that can source text from specific LLMs. |
| Outcome: | The proposed tool can source text from specific LLMs, such as GPT-2, OPT, LLaMA, and others. |
Analyzing the Perceived Severity of Cybersecurity Threats Reported on Social Media (N19-1)
Copied to clipboard
| Challenge: | 6,000 tweets describe software vulnerabilities, which are shared across a range of websites and social media platforms. |
| Approach: | They propose a method to link software vulnerabilities reported in tweets to CVEs in the National Vulnerability Database (NVD) a Precision@50 of 0.86 is achieved when forecasting high severity vulnerabilities, they show . |
| Outcome: | The proposed method outperforms baseline methods based on tweet volume and the language used to describe them online. |
Inferring about fraudulent collusion risk on Brazilian public works contracts in official texts using a Bi-LSTM approach (2020.findings-emnlp)
Copied to clipboard
Marcos Lima, Roberta Silva, Felipe Lopes de Souza Mendes, Leonardo R. de Carvalho, Aleteia Araujo, Flavio de Barros Vidal
| Challenge: | Public works procurements are a preferred field for collusion and fraud in Brazil . current methods of fraud detection use structured data to classification and usually do not involve annotated data. |
| Approach: | They propose to use public works procurements to classify risky entries using a dataset of 15,132,968 textual entries of which 1,907 are annotated. |
| Outcome: | The proposed datasets show that both bottleneck deep neural network and biLSTM are competitive compared with classical classifiers and achieve better precision (93.0% and 92.4%, respectively). |
MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) generate plausible but factually incorrect outputs, posing serious risks to patient safety and clinical decision-making. |
| Approach: | They propose a benchmark for medical hallucination detection using 10,000 question-answer pairs derived from PubMedQA. |
| Outcome: | The proposed model achieves an F1 score as low as 0.625 for detecting 'hard' category hallucinations. |
Smart “Chef”: Verifying the Effect of Role-based Paraphrasing for Aspect Term Extraction (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Aspect Term Extraction (ATE) is a task of automatically extracting aspect terms from sentences. |
| Approach: | They propose to automatically rewrite sentences from virtual experts with different roles . they leverage ChatGPT to determine virtual experts in the considered domains . |
| Outcome: | The proposed method can be used to expand the predictions obtained on the original sentences without retraining or fine-tuning the baseline extractors. |
Assessing the Efficacy of Grammar Error Correction: A Human Evaluation Approach in the Japanese Context (2024.lrec-main)
Copied to clipboard
| Challenge: | Using an automatic annotation toolkit, we evaluated the performance of the sequence tagging grammar error detection and correction model (SeqTagger) using Japanese university students’ writing samples. |
| Approach: | They evaluated the performance of the state-of-the-art sequence tagging grammar error detection and correction model using Japanese university students’ writing samples. |
| Outcome: | The proposed model shows a high precision but conservativeness in error detection and correction. |
Dialogue is Better Than Monologue: Instructing Meidcal LLMs via Strategic Conversations (2026.findings-eacl)
Copied to clipboard
Zijie Liu, Xinyu Zhao, Jie Peng, Jinhao Duan, Zhuangdi Zhu, Qingyu Chen, Kaidi Xu, Xia Hu, Tianlong Chen
| Challenge: | Existing tuning methods for medical AI models are monologue-based . existing benchmarks are based on licensing exams or research articles . |
| Approach: | They propose a benchmark to expose limitations of monologue-based tuning for medical AI models . they use a large dialogue dataset to capture stepwise diagnostic reasoning . |
| Outcome: | The proposed model outperforms monologue-tuned models on a medical question answering task and improves accuracy on standard medical QA benchmarks. |
Fine-Grained Error Analysis and Fair Evaluation of Labeled Spans (2022.lrec-1)
Copied to clipboard
| Challenge: | Annotations with incorrect label or boundaries count as two errors instead of one, despite being closer to the target annotation than false positives or false negatives. |
| Approach: | They propose an algorithm for error identification in flat and multi-level annotations and propose a procedure for calculating meaningful precision, recall, and F1-scores based on the more fine-grained error types. |
| Outcome: | The proposed procedure prevents double penalties and allows for a more detailed error analysis, providing more insight into the actual weaknesses of a system. |
A Silver Standard Corpus of Human Phenotype-Gene Relations (N19-1)
Copied to clipboard
| Challenge: | Existing tools for phenotype-gene relations extraction require annotated corpus, which requires manual effort and time. |
| Approach: | They propose to generate a silver standard corpus of human phenotype and gene annotations and their relations using Named-Entity Recognition tools. |
| Outcome: | The proposed corpus was generated with Named-Entity Recognition tools with a precision of 87.01%. |
GenRES: Rethinking Evaluation for Generative Relation Extraction in the Era of Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing relation extraction methods rely on exact matching with human-annotated reference relations, while GRE methods produce diverse and semantically accurate relations. |
| Approach: | They propose a multi-dimensional assessment of relation extraction methods using human-annotated reference relations. |
| Outcome: | The proposed method is consistent with human preferences for RE quality. |
Valency-Augmented Dependency Parsing (D18-1)
Copied to clipboard
| Challenge: | valency analysis is a complex task that requires a large number of subcategorizations, such as the number and types of syntactic dependents. |
| Approach: | They propose a parsing approach that explicitly models the number and types of syntactic dependents as valency patterns and a probabilistic model for tagging them. |
| Outcome: | The proposed approach outperforms the state-of-the-art labeled attachment score on 53 treebanks representing 41 languages and outperformed the previous state- of-the art labeles by 0.7. |
How Does Response Length Affect Long-Form Factuality (2025.findings-acl)
Copied to clipboard
| Challenge: | Despite growing attention to LLM factuality, the effect of response length on factual accuracy remains underexplored. |
| Approach: | They propose an automatic and bi-level long-form factuality evaluation framework which achieves high agreement with human annotations while being cost-effective. |
| Outcome: | The proposed framework achieves high agreement with human annotations while being cost-effective. |
pEBR: A Probabilistic Approach to Embedding Based Retrieval (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing embedding-based retrieval systems rely on heuristic and suboptimal cutoffs for item retrieval. |
| Approach: | They propose a probabilistic Embedding-Based Retrieval framework that learns a shared semantic representation space for both queries and items. |
| Outcome: | The proposed framework improves retrieval precision and recall, and ablation studies show it captures the differences between head-to-tail queries. |
A Multi- versus a Single-classifier Approach for the Identification of Modality in the Portuguese Language (L18-1)
Copied to clipboard
| Challenge: | Comparative study of two different approaches to build an automatic classification system for Modality values in the Portuguese language. |
| Approach: | They propose to use a single multi-class classifier with the full Portuguese language dataset that includes eleven modal verbs and a weighted average approach to build different classifiers for each verb. |
| Outcome: | The proposed system is based on a Portuguese language dataset with 11 modal verbs and two different classifiers, one for each verb. |
Mining Health-related Cause-Effect Statements with High Precision at Large Scale (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for assessing the health relatedness of phrases and sentences are slower and less effective than state-of-the-art medical entity linkers. |
| Approach: | They propose a termhood score that achieves 69% recall at over 90% precision on a web dataset with cause-effect statements. |
| Outcome: | The proposed method achieves 69% recall at over 90% precision on a web dataset with cause-effect statements. |
No Gestures Left Behind: Learning Relationships between Spoken Language and Freeform Gestures (2020.findings-emnlp)
Copied to clipboard
| Challenge: | a study of spoken language and co-speech gestures shows that it is important to model the long tail of the language-gesture distribution. |
| Approach: | They propose a method that combines adversarial learning with importance sampling to strike a balance between precision and coverage. |
| Outcome: | The proposed method outperforms state-of-the-art methods for gesture generation. |
Language-Driven Region Pointer Advancement for Controllable Image Captioning (2020.coling-main)
Copied to clipboard
| Challenge: | Controllable Image Captioning is a recent sub-task of Image Captions wherein constraints are placed on which regions in an image should be described in the generated natural language caption. |
| Approach: | They propose a method for predicting the timing of region pointer advancement by treating the advancement step as a natural part of the language structure via a NEXT-token. |
| Outcome: | The proposed method agrees with ground-truth timing in the Flickr30k Entities test data with a precision of 86.55% and a recall of 97.92%. |
Towards an argumentative content search engine using weak supervision (C18-1)
Copied to clipboard
| Challenge: | Existing work focused on detecting claims within a small set of documents . however, pinpointing relevant claims within massive unstructured corpora, received little attention. |
| Approach: | They propose to use a weak signal to develop a query for claim–sentence detection using a large text corpus. |
| Outcome: | The proposed system outperforms previous results in terms of precision and coverage. |
Knowledge Enhanced Fine-Tuning for Better Handling Unseen Entities in Dialogue Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for dialogue generation use an external knowledge base to generate appropriate responses. |
| Approach: | They propose to use an external knowledge base to generate appropriate responses for unseen entities. |
| Outcome: | Experiments on two dialogue corpus show that pre-trained models perform poorly with unseen entities. |
Momoka-RAG: MCTS-Organized Mapping of Knowledge Associations for Long-Document Retrieval Augmented Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks that rely on fixed-length chunking are unsuitable for long-document tasks due to their passive and mechanical approach to knowledge structure. |
| Approach: | They propose a framework that utilizes Monte Carlo Tree Search to proactively uncover connections among chunks and construct optimal semantic information paths with the objective of completing semantic relationships. |
| Outcome: | The proposed framework achieves higher precision while maintaining competitive recall compared to other RAG frameworks. |
Contrastive Token Learning with Similarity Decay for Repetition Suppression in Machine Translation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is pivotal for crosslingual conversation and trade . traditional solutions that penalize text redundancy or token reoccurrence have shown limited efficacy . |
| Approach: | They propose an algorithm that modulates suppression of tokens dynamically, informed by attention weights and inter-token distances. |
| Outcome: | The proposed algorithm outperforms existing methods in precision and generalizability. |
Guaranteeing Knowledge Integration with Joint Decoding for Retrieval-Augmented Generation (2026.acl-long)
Copied to clipboard
Zhengyi Zhao, Shubo Zhang, Zezhong Wang, Yuxi Zhang, Huimin Wang, Yutian Zhao, Yefeng Zheng, Binyang Li, Kam-Fai Wong, Xian Wu
| Challenge: | Retrieval-Augmented Generation (RAG) provides access to external knowledge, but current research focuses on retrieval quality and 'integration bottleneck' . |
| Approach: | They propose a framework that explicitly decouples reasoning from evidence integration by generating an 'Inner-Answer' and a 'Refer-Aswer" they propose 'a joint decoding mechanism that dynamically fuses the logical coherence of the Inner-Andswer with the factual precision of the Refer-Adswer at the token level' |
| Outcome: | The proposed framework improves accuracy by 12.1% and reduces hallucinations by 16.3% on five QA benchmarks. |
Exploratory Neural Relation Classification for Domain Knowledge Acquisition (C18-1)
Copied to clipboard
| Challenge: | Existing methods for relation classification are limited and lack of low-frequency relations in specific domains. |
| Approach: | They propose a method to learn a classifier on pre-defined relations and discover new relations expressed in texts. |
| Outcome: | The proposed method can classify entities into a finite set of relations and discover relations with high precision and recall. |
The Two Paradigms of LLM Detection: Authorship Attribution vs Authorship Verification (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for detecting texts generated by large language models are disputed . authors argue that there are limitations in the current technology . |
| Approach: | They propose to make LLM detectors robust against domain shifts and build benchmarks . they argue that the limitations lie elsewhere, and open the realm of authorship analysis technology . |
| Outcome: | The proposed method systematically analyzes the benchmarks and validates it using state-of-the-art detectors. |
Open Information Extraction from Conjunctive Sentences (C18-1)
Copied to clipboard
| Challenge: | Recent work has highlighted the lack of proper conjunction processing as the most significant source of missed yield in Open IE. |
| Approach: | They develop a coordination analyzer that searches over hierarchical conjunct boundaries and uses a language model to score conjunctions. |
| Outcome: | The proposed system performs extraction over the simple sentences identified by CALM to obtain up to 1.8x yield with a moderate increase in precision compared to extractions from original sentences. |
Divide and Conquer: Legal Concept-guided Criminal Court View Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for creating rationales for criminal cases do not pay enough attention to the important legal concepts. |
| Approach: | They propose a legal concept-guided court view generation framework that generates rationales based on predicted legal concepts . they first divide the court view into sub-views, then employ a solver and verifier to generate and select rationale. |
| Outcome: | The proposed model generates coherent and coherent court views on a real-world criminal case dataset. |
PREE: Towards Harmless and Adaptive Fingerprint Editing in Large Language Models via Knowledge Prefix Enhancement (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing black-box fingerprinting techniques rely on overfitting high-perplexity trigger patterns . experimental results show that model editing in the fingerprint domain exhibits unique advantages . |
| Approach: | They propose a prefix-enhanced fingerprint editing framework that encodes copyright information into parameter offsets through dual-channel knowledge edit to achieve covert embedding of fingerprint features. |
| Outcome: | The proposed model editing framework achieves 90% trigger precision in mainstream architectures . the proposed model editor achieves the 90% accuracy in mainstream models . |
Multimedia Event Extraction with LLM Knowledge Editing (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing multimodal event extraction methods focus on weakly aligning features from wellpretrained unimodal encoders, resulting in redundant feature perception. |
| Approach: | They propose a multimodal event extraction strategy with a redundant feature selection mechanism that enhances event understanding ability of multimodal large language models. |
| Outcome: | The proposed method outperforms the state-of-the-art (SOTA) baselines on the M2E2 benchmark. |
Precision or Recall? An Analysis of Image Captions for Training Text-to-Image Generation Model (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in text-to-image models have demonstrated remarkable capabilities in image synthesis. |
| Approach: | They analyze the critical role of caption precision and recall in text-to-image model training. |
| Outcome: | The proposed model trains with synthetic captions that show similar behavior to those trained on human-annotated captions. |
Adapting Open Domain Fact Extraction and Verification to COVID-FACT through In-Domain Language Modeling (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to verify scientifically false online information are limited by the lack of training data in the scientific domain. |
| Approach: | They propose an in-domain language modeling method for fact extraction and verification systems . they use SCIFACT to extract scientifically false online information . |
| Outcome: | The proposed method improves accuracy 30% on SCIFACT dataset . state-of-the-art model achieves only 46.6% precision, which is hard to be trusted for users. |
Sanitizing Large Language Models in Bug Detection with Data-Flow (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have potential in code reasoning tasks but the hallucination effect can compromise the reliability of bug reports. |
| Approach: | They propose a new schema of bug detection that enforces LLMs to emit data-flow paths in few-shot chain-of-thought prompting and validates them via the program-property decomposition. |
| Outcome: | The proposed approach achieves 91.03% precision and 74.00% recall upon synthetic benchmarks and boosts precision by 21.99% with the sanitization. |
How Many Ratings per Item are Necessary for Reliable Significance Testing? (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for estimating model reliability are based on a few output responses per item. |
| Approach: | They propose a method to determine whether an existing dataset has enough responses per item to assure reliable null hypothesis statistical testing. |
| Outcome: | The proposed method can help researchers make better decisions about how to collect data for AI evaluation. |
Regular Expression Guided Entity Mention Mining from Noisy Web Data (D18-1)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a subtask of the broader problem of Information Extraction (IE) from text. |
| Approach: | They propose a framework that uses Regular Expressions to identify entities from web data . they combine expressive power of REs with ability of deep learning to learn from large data a human expert is asked to label a small set of documents . |
| Outcome: | The proposed framework achieves impressive accuracy while requiring modest human effort. |
Automatic Wordnet Mapping: from CoreNet to Princeton WordNet (L18-1)
Copied to clipboard
| Challenge: | Existing mappings focus on identifying the semantic categories of CoreNet, but not the word senses. |
| Approach: | They propose to map the word senses of CoreNet into Princeton WordNet synsets by lexical relations by a taxonomy. |
| Outcome: | The proposed mapping bridging the gap between CoreNet and WordNet shows that the word senses of CoreNet are mapped with precision of 91.2%. |
AELC: Adaptive Entity Linking with LLM-Driven Contextualization (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Entity linking (EL) focuses on associating ambiguous mentions in text with corresponding entities in a knowledge graph. |
| Approach: | Entity linking (EL) focuses on associating ambiguous mentions in text with corresponding entities in a knowledge graph. |
| Outcome: | Experiments on four public benchmark datasets show that AELC achieves state-of-the-art performance. |
Life is a Circus and We are the Clowns: Automatically Finding Analogies between Situations and Processes (2022.emnlp-main)
Copied to clipboard
| Challenge: | Analogy-making gives rise to reasoning, abstraction, flexible categorization and counterfactual inference – abilities that current AI systems lack. |
| Approach: | They propose an interpretable, scalable algorithm that extracts analogies from a pair of natural language procedural texts and finds a mapping between the different domains based on relational similarity. |
| Outcome: | The proposed algorithm can extract analogies from a large dataset and achieve 79% precision. |
A Top-down Graph-based Tool for Modeling Classical Semantic Maps: A Case Study of Supplementary Adverbs (2025.naacl-long)
Copied to clipboard
| Challenge: | Semantic map models (SMMs) construct a network-like conceptual space from cross-linguistic instances or forms based on the connectivity hypothesis. |
| Approach: | They propose a graph-based algorithm that automatically generates conceptual spaces and SMMs in a top-down manner. |
| Outcome: | The proposed model is compared with human annotations and other automated methods on cross-linguistic supplementary adverbs. |
Mitigating Hallucinations in Large Vision-Language Models via Summary-Guided Decoding (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) generate detailed and coherent responses from visual inputs but are prone to generate hallucinations due to an over-reliance on language priors. |
| Approach: | They propose a method that reduces the text context and controls only the image-related POS tokens to maintain text quality by reducing the text contextualization. |
| Outcome: | The proposed method achieves state-of-the-art performance on object hallucination benchmarks and achieves Pareto optimality among the existing methods. |
SAC-KG: Exploiting Large Language Models as Skilled Automatic Constructors for Domain Knowledge Graph (2024.acl-long)
Copied to clipboard
| Challenge: | Existing KG construction methods rely on human intervention to attain qualified KGs, which severely hinders the practical application of domain KG. |
| Approach: | They propose a general KG construction framework that uses large language models as "S**killed" A**utomatic C**onstructors for domain knowledge (G**raph) |
| Outcome: | The proposed framework generates specialized multi-level knowledge graphs at the scale of over one million nodes and achieves 89.32% precision rate compared to state-of-the-art methods. |
Improving Hypernymy Extraction with Distributional Semantic Classes (L18-1)
Copied to clipboard
| Challenge: | Existing methods for extracting hypernyms focus on the acquisition of binary hypernies . |
| Approach: | They propose a distributionally-induced semantic class for extracting hypernyms . they also use distributional semantics to induce sense-aware semantic classes . |
| Outcome: | The proposed method improves the quality of the hypernymy extraction in terms of precision and recall. |
TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoning (2025.findings-naacl)
Copied to clipboard
| Challenge: | Current Large Language Models lack ability to understand table structures and apply precise numerical reasoning. |
| Approach: | They propose a tool-augmented reasoning framework for table-based tasks that integrates LLMs with specialized tools. |
| Outcome: | The proposed framework improves on the TOOLTAB dataset, a benchmark for LLMs in table–tool integration. |
Improving Multi-label Malevolence Detection in Dialogues through Multi-faceted Label Correlation Enhancement (2022.acl-long)
Copied to clipboard
| Challenge: | Current methods for detecting dialogue malevolence neglect label correlation. |
| Approach: | They propose to crowdsource a multi-label dataset for detecting malevolent dialogue responses and a model with label correlation enhanced CRF to measure the correlation between malevolence and negative emotions. |
| Outcome: | The proposed model outperforms the best performing baseline method on precision, recall, F1, and Jaccard score by 16.1%, 11.9%, 12.0%, and 6.1% on malevolence. |
Vicinal Risk Minimization for Few-Shot Cross-lingual Transfer in Abusive Language Detection (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for few-shot cross-lingual transfer learning are limited in target languages due to the scarcity of resources. |
| Approach: | They propose a method which interpolates pairs of instances based on the angle of their representations and propose augmentation methods to enhance few-shot cross-lingual abusive language detection. |
| Outcome: | The proposed method improves few-shot cross-lingual abusive language detection in seven languages typologically distinct from English and three different domains. |
Can Generative Pre-trained Language Models Serve As Knowledge Bases for Closed-book QA? (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work is limited in using small benchmarks with high test-train overlaps. |
| Approach: | They construct a dataset of closed-book QA using SQuAD and investigate the performance of BART. |
| Outcome: | Experiments show that pre-trained language models can achieve high performance on closed-book QA tasks. |
We Are What We Repeatedly Do: Improving Long Context Instruction Following (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language model context lengths have increased by at least 1000 in the past seven years . however, longer contexts pose challenges to system instruction following . |
| Approach: | They propose to formalize verifiable instructions to evaluate model compliance . they implement and evaluate six mitigation strategies to enhance instruction compliance in extended contexts. |
| Outcome: | The proposed model performs better in long contexts than in natural language models. |
Transparent Human Evaluation for Image Captioning (2022.naacl-main)
Copied to clipboard
Jungo Kasai, Keisuke Sakaguchi, Lavinia Dunagan, Jacob Morrison, Ronan Le Bras, Yejin Choi, Noah A. Smith
| Challenge: | Recent work has demonstrated that image captioning is a complex task that requires a large amount of human input. |
| Approach: | They develop a human evaluation protocol for image captioning models based on machine- and human-generated captions on the MSCOCO dataset. |
| Outcome: | The proposed model improves CLIPScore, a recent metric that uses image features, and improves human judgments because it is more sensitive to recall. |
CANarEx: Contextually Aware Narrative Extraction for Semantically Rich Text-as-data Applications (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Narrative modelling is a field of active research that conceptualizes narratives as connected entity chains. |
| Approach: | They propose an alternative narrative extraction approach using semantic role labeling to extract tuples from text, then dimensionality reduction to reduce the space of entities and connections separately. |
| Outcome: | The proposed approach improves on a text-as-data task and improves accuracy and recall. |
CBT-LLM: A Chinese Large Language Model for Cognitive Behavioral Therapy-based Mental Health Question Answering (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in artificial intelligence highlight the potential of language models in psychological health support. |
| Approach: | They propose a method to enhance the precision and efficacy of psychological support through large language models. |
| Outcome: | The proposed model generates professional and structured responses in Chinese psychological health Q&A tasks, showcasing its practicality and quality. |
From Stance to Concern: Adaptation of Propositional Analysis to New Tasks and Domains (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing paradigms for propositional analysis use stances and concerns to generate explanatory representations. |
| Approach: | They propose a generalized paradigm for adaptation of propositional analysis to new tasks and domains by using an analogy between stances and concerns. |
| Outcome: | The proposed model yields 231% improvement in recall over baseline, with only 10% loss in precision. |
QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models (2025.emnlp-main)
Copied to clipboard
Jiajun Zhou, Yifan Yang, Kai Zhen, Ziyue Liu, Yequan Zhao, Ershad Banijamali, Athanasios Mouchtaris, Ngai Wong, Zheng Zhang
| Challenge: | Large Language Models (LLMs) are quantized to lower precision to reduce memory cost and latency in inference. |
| Approach: | They propose a quantized zeroth-order framework for fine-tuning Large Language Models (LLMs) using low-precision forward passes. |
| Outcome: | The proposed method achieves comparable results to first-order methods in FP8 and superior accuracy in INT8 and INT4 training. |
Discriminative Learning of Open-Vocabulary Object Retrieval and Localization by Negative Phrase Augmentation (D18-1)
Copied to clipboard
| Challenge: | Existing methods for object detection only handle pre-specified classes, requiring large amounts of visual samples for training. |
| Approach: | They propose a method to retrieve and localize objects specified by a textual query from one million images in 0.5 seconds with high precision. |
| Outcome: | The proposed method can retrieve and localize objects specified by a textual query from one million images in 0.5 seconds with high precision. |
Beyond the Surface: A Solution-Aware Retrieval Model for Competition-level Code Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing retrieval models emphasize surface-level semantic similarity, neglecting deeper solution-level logical similarities. |
| Approach: | They propose a solution-aware ranking model empowered by synthetic data for competitive programming tasks. |
| Outcome: | The proposed ranking model outperforms existing retrieval models in precision and recall metrics. |
On Evaluating the Integration of Reasoning and Action in LLM Agents with Database Question Answering (2024.findings-naacl)
Copied to clipboard
| Challenge: | a new study evaluates how Large Language Models interact with a SQL interpreter . the model is limited in context and is stochastic, making it less suited for tasks requiring high precision and extensive computations. |
| Approach: | They propose and evaluate two interaction strategies to evaluate how LLMs interact with a SQL interpreter. |
| Outcome: | The proposed framework improves the accuracy and reliability of the evaluations. |
Fusion meets Function: The Adaptive Selection-Generation Approach in Event Argument Extraction (2025.coling-main)
Copied to clipboard
| Challenge: | Event Argument Extraction is a critical subtask of Event Extraction, focused on identifying event arguments within text. |
| Approach: | They propose a Fusion Selection-Generation-Based Approach that merges selective and generative methods to enhance argument extraction accuracy. |
| Outcome: | The proposed method improves on the RAMS and WikiEvents, while preserving the unique characteristics of both methods. |
Tackling Distractor Documents in Multi-Hop QA with Reinforcement and Curriculum Learning (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing work on retrieval-augmented generation systems has shown that retrievers exhibit imperfect recall and precision, limiting downstream performance. |
| Approach: | They propose a retrieval-augmented generation model that generates answers from larger sets of retrieved contexts. |
| Outcome: | The proposed model generates answers and cites relevant information from larger sets of retrieved contexts. |
Weakly Supervised Multilingual Causality Extraction from Wikipedia (D19-1)
Copied to clipboard
| Challenge: | Existing methods for extracting causality knowledge from Wikipedia are lacking in this area. |
| Approach: | They propose a method for extracting causality knowledge from Wikipedia . they exploit the multilinguality of Wikipedia and the ability to translate to multiple languages . |
| Outcome: | The proposed method achieves precision and recall above 98% and 64%, respectively. |
Normalized Narrow Jump To Conclusions: Normalized Narrow Shortcuts for Parameter Efficient Early Exit Transformer Prediction (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that shortcutting pre-trained transformers to final representations can be cheaper and improve model performance but can also increase computational costs. |
| Approach: | They propose Narrow Jump to Conclusions and Normalized Narrow jump to conclusions that reduce shortcut parameter count by over 97%. |
| Outcome: | The proposed approaches outperform Identity shortcuts at early stages and offer stable precision from all transformer block levels for GPT-2-XL, Phi3-Mini and Llama2-7B models. |
The Chinese Remainder Theorem for Compact, Task-Precise, Efficient and Secure Word Embeddings (2021.eacl-main)
Copied to clipboard
| Challenge: | a new method for compressing word vector embeddings into integers is being developed . a high precision approach to compressing words into integer results in negligible performance gains . |
| Approach: | They propose a method for compressing word vector embeddings into integers using the Chinese Reminder Theorem. |
| Outcome: | The proposed method speeds up addition by 48.27% and compresses GloVe word embedding libraries by 25.86%. |
OpenCeres: When Open Information Extraction Meets the Semi-Structured Web (N19-1)
Copied to clipboard
| Challenge: | Open Information Extraction (OpenIE) is a problem of extracting triples from natural language text whose predicate relations are not aligned to any pre-defined ontology. |
| Approach: | They propose an open-source method to extract triples from semi-structured websites . they use a semi-supervised label propagation technique to create training data for relations . |
| Outcome: | The proposed method extracts over 2 million triples from 31 websites in the movie vertical. |
Chain-of-Rank: Enhancing Large Language Models for Domain-Specific RAG in Edge Device (2025.findings-naacl)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) is valuable in specialized domains where precision is critical. |
| Approach: | They propose a chain-of-rank algorithm which allows LLMs to access a target domain early via finetuning. |
| Outcome: | The proposed method achieves state-of-the-art in benchmarks and analyzes its efficacy. |
Evaluation of Dictionary Creating Methods for Finno-Ugric Minority Languages (L18-1)
Copied to clipboard
| Challenge: | a project aims to provide linguistically based support for small Finno-Ugric (FU) digital communities to generate online content and revitalize the digital functions of some FU minority languages. |
| Approach: | They evaluate bilingual dictionary building methods for six small fino-ugric minority languages . they use Wikipedia title pairs extracted via inter-language links and Wiktionary-based methods . |
| Outcome: | The proposed methods proved that standard lexicon building methods are low for under-resourced languages. |
A Finite-State Morphological Analyser for Evenki (2020.lrec-1)
Copied to clipboard
| Challenge: | Evenki is a language with rich morphology, therefore a morphological analyser is highly desirable for processing Evenki texts. |
| Approach: | They propose to use a morphological analyser for Evenki to analyze half of the corpus . they evaluate the morphology of available corpora and estimate accuracy, recall and F-score . |
| Outcome: | The proposed morphological analyser can analyse less than a half of the available corpora on Evenki . it is based on the Helsinki Finite-State Transducer toolkit (HFST). |
MIsA: Multilingual “IsA” Extraction from Corpora (L18-1)
Copied to clipboard
| Challenge: | In this paper, we present a collection of hypernymy relations extracted from the Wikipedia corpus in five languages. |
| Approach: | They present a collection of hypernymy relations extracted from the Wikipedia corpus in five languages . they use existing or newly defined lexico-syntactic patterns to extract hyperniyms . |
| Outcome: | The proposed tool is based on a dictionary extracted from the full Wikipedia corpus. |
MixKVQ: Query-Aware Mixed-Precision KV Cache Quantization for Long-Context Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing low-bit quantization methods often exhibit severe performance degradation on complex reasoning tasks. |
| Approach: | They propose a plug-and-play method that uses a key channel's intrinsic quantization difficulty and relevance to the query to identify and preserve critical key channels that need higher precision. |
| Outcome: | Experiments on complex reasoning datasets show that the proposed method outperforms low-bit methods at a substantially reduced memory footprint. |
Transforming Complex Sentences into a Semantic Hierarchy (P19-1)
Copied to clipboard
| Challenge: | Existing approaches for recursively splitting and rephrasing complex English sentences into a semantic hierarchy of simplified sentences are lacking. |
| Approach: | They propose a method for recursively splitting and rephrasing complex English sentences into a semantic hierarchy of simplified sentences. |
| Outcome: | The proposed approach outperforms state-of-the-art approaches in MT and information extraction tasks. |
Semi-Supervised Dialogue Abstractive Summarization via High-Quality Pseudolabel Selection (2024.naacl-long)
Copied to clipboard
| Challenge: | Semi-supervised dialogue summarization (SSDS) leverages model-generated summaries to reduce reliance on human-labeled data. |
| Approach: | They propose a scoring approach that encapsulates three primary dimensions of summarization model quality. |
| Outcome: | The proposed method reduces reliance on human-labeled data and improves the performance of summarization models. |
Extracting a Knowledge Base of COVID-19 Events from Social Media (2022.coling-1)
Copied to clipboard
| Challenge: | a flood of COVID-19 related information has appeared on social media since December 2019 . this includes reports on public figures who have tested positive/negative for the virus . |
| Approach: | They construct a corpus of 10,000 tweets with annotated public reports of five COVID-19 events, using slot-filling questions to fill in slots. |
| Outcome: | The proposed method can be quickly applied to develop knowledge bases for new domains in response to emerging crises, including natural disasters or future disease outbreaks. |
FairPrism: Evaluating Fairness-Related Harms in Text Generation (2023.acl-long)
Copied to clipboard
Eve Fleisig, Aubrie Amstutz, Chad Atalla, Su Lin Blodgett, Hal Daumé III, Alexandra Olteanu, Emily Sheng, Dan Vann, Hanna Wallach
| Challenge: | FairPrism dataset provides a framework for measuring and mitigating fairness-related harms caused by AI text generation systems. |
| Approach: | They propose a dataset of 5,000 examples of AI-generated English text with detailed human annotations covering a diverse set of harms relating to gender and sexuality. |
| Outcome: | FairPrism is a dataset of 5,000 examples of AI-generated English text with detailed human annotations covering harms relating to gender and sexuality. |
Logit Separability-Driven Samples and Multiple Class-Related Words Selection for Advancing In-Context Learning (2025.naacl-long)
Copied to clipboard
| Challenge: | Effective organization of in-context learning (ICL) demonstrations is key to improving the quality of large language models (LLMs). |
| Approach: | They propose a logit separability-based method that integrates multiple class-related words into each sample-label pair to improve LLM understanding. |
| Outcome: | The proposed method improves ICL performance by providing clearer instructions and richer label information. |
arXiv2Table: Toward Realistic Benchmarking and Evaluation for LLM-Based Literature-Review Table Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Literature review tables are essential for summarizing and comparing collections of scientific papers. |
| Approach: | They propose to generate a database of literature review tables from a pool of papers and to model retrieval noise via semantically related but out-of-scope distractor papers verified by human annotators. |
| Outcome: | The proposed method improves over strong baselines while the absolute scores remain modest, underscoring the task’s difficulty. |
An Effectiveness Metric for Ordinal Classification: Formal Properties and Experimental Results (2020.acl-main)
Copied to clipboard
| Challenge: | Existing Ordinal Classification metrics ignore the ordering between items or assume additional information. |
| Approach: | They propose a Closeness Evaluation Measure for Ordinal Classification based on Measurement Theory and Information Theory. |
| Outcome: | The proposed metric captures quality aspects from different traditional tasks simultaneously. |
An Empirical Methodology for Detecting and Prioritizing Needs during Crisis Events (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Social media platforms such as Twitter contain a vast amount of information about the general public’s needs. |
| Approach: | They propose to use Twitter to extract a list of needed resources and detecting sentences that specify who-needs-what resources. |
| Outcome: | The proposed methods achieve 0.64 precision on a set of 1,000 annotated tweets and achieve 0.68 F1-score. |
MADD: Multi-Agent Drug Discovery Orchestra (2025.findings-emnlp)
Copied to clipboard
Gleb Vitalevich Solovev, Alina Borisovna Zhidkovskaya, Anastasia Orlova, Nina Gubina, Anastasia Vepreva, Rodion Golovinskii, Ilya Tonkii, Ivan Dubrovsky, Ivan Gurev, Dmitry Gilemkhanov, Denis Chistiakov, Timur A. Aliev, Ivan Poddiakov, Galina Zubkova, Ekaterina V. Skorb, Vladimir Vinogradov, Alexander Boukhanovsky, Nikolay Nikitin, Andrei Dmitrenko, Anna Kalyuzhnaya, Andrey Savchenko
| Challenge: | Recent advances in artificial intelligence have limited access to wet-lab tools for hit identification . multi-agent systems combine interpretability of LLMs with precision of specialized models and tools . |
| Approach: | They propose a multi-agent system that builds and executes customized hit identification pipelines from natural language queries. |
| Outcome: | The proposed system reduces the complexity of traditional screening methods and improves efficiency. |
Document-Level N-ary Relation Extraction with Multiscale Representation Learning (N19-1)
Copied to clipboard
| Challenge: | Existing work on cross-sentence relation extraction is limited to three consecutive sentences, which severely limits recall. |
| Approach: | They propose a multiscale neural architecture for document-level n-ary relation extraction that combines representations learned over various text spans throughout the document and across the subrelation hierarchy. |
| Outcome: | The proposed system outperforms existing methods on biomedical machine reading. |
∞-former: Infinite Memory Transformer (2022.acl-long)
Copied to clipboard
| Challenge: | Several efficient transformers have been proposed, but they all have a finite memory capacity and are forced to drop old information. |
| Approach: | They propose an unbounded long-term memory extension that extends the vanilla transformer by using a continuous-space attention mechanism to attend over the long-time memory. |
| Outcome: | The proposed model can model arbitrarily long contexts while keeping the computation budget fixed. |
MIDGARD: Self-Consistency Using Minimum Description Length for Structured Commonsense Reasoning (2024.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to generate a reasoning graph from natural language input suffer from error propagation due to autoregressive nature and single-pass-based decoding. |
| Approach: | They propose a method that uses minimum description length to identify consistent properties among different graph samples generated by large language models. |
| Outcome: | The proposed method outperforms previous approaches for generating reasoning graphs from natural language input using large language models. |
FocusQA: Open-Domain Question Answering with a Context in Focus (2022.findings-emnlp)
Copied to clipboard
Gianni Barlacchi, Ivano Lauriola, Alessandro Moschitti, Marco Del Tredici, Xiaoyu Shen, Thuy Vu, Bill Byrne, Adrià de Gispert
| Challenge: | a new method for question answering with a context in focus simulates a free interaction with QA systems. |
| Approach: | They introduce question answering with a cotext in focus task that simulates a free interaction with QA systems. |
| Outcome: | The proposed model outperforms state-of-the-art models for question answering with a context in focus up to 21.3% absolute points. |
FinGEAR: Financial Mapping-Guided Enhanced Answer Retrieval (2025.findings-emnlp)
Copied to clipboard
| Challenge: | FinGEAR provides a retrieval framework tailored to financial documents . standard retrieval-augmented generation models underuse financial disclosures . |
| Approach: | FinGEAR combines a finance lexicon for Item-level guidance and hierarchical indices for within-Item search. |
| Outcome: | FinGEAR improves accuracy and accuracy on 10-Ks with a FinQA dataset. |
Learning to Extract Structured Entities Using Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Language Models (LMs) play a pivotal role in extracting structured information from unstructured text. |
| Approach: | They propose to reformulate the task to be entity-centric, enabling the use of diverse metrics that can provide more insights from various perspectives. |
| Outcome: | The proposed model outperforms baselines and human evaluations on the extracted entities. |
A Dataset of Translational Equivalents Built on the Basis of plWordNet-Princeton WordNet Synset Mapping (2020.lrec-1)
Copied to clipboard
| Challenge: | a dataset of 11,000 Polish-English translational equivalents is presented . the dataset is a novum in the wordnet domain and can facilitate the precision of bilingual NLP tasks. |
| Approach: | They present a dataset of Polish-English translational equivalents linked by three types of equivalence links. |
| Outcome: | The proposed dataset contains 11,000 Polish-English translational equivalents . the resulting subsets are based on a manual annotation process and a set of formal features . |
Extreme Zero-Shot Learning for Extreme Text Classification (2022.naacl-main)
Copied to clipboard
| Challenge: | Experimental results show that MACLR achieves superior performance compared to other baseline methods. |
| Approach: | They propose to pre-train Transformer-based encoders with self-supervised contrastive losses to learn the semantic embeddings of instances and labels with raw text. |
| Outcome: | The proposed method improves on the EZ-XMC model with a limited number of ground-truth positive pairs. |
Bayes Test of Precision, Recall, and F1 Measure for Comparison of Two Natural Language Processing Models (P19-1)
Copied to clipboard
| Challenge: | Existing t-tests for cross-validation (CV) are inappropriate for model comparison . existing t tests for cross validation (CV), such as 52 CV t test and F ttest, are inadequate . |
| Approach: | They propose to use a block-regularized 32 CV to compare two NLP models . they calibrate the posterior distributions of P, R, and F1 and derive an accurate interval estimation of P and R . |
| Outcome: | The proposed model could regularize the difference in certain frequency distributions over linguistic units and yield stable estimators of P, R, and F1. |
Conspiracy Theories and Where to Find Them on TikTok (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies on TikTok's potential to promote and amplify harmful content have not been conducted. |
| Approach: | They analyze a longitudinal dataset of 1.5M videos shared in the U.S. over three years and evaluate the effects of TikTok’s Creativity Program for monetization. |
| Outcome: | The proposed model achieves high precision in detecting harmful content, but its overall performance is comparable to fine-tuned traditional models such as RoBERTa. |
HeLI-OTS, Off-the-shelf Language Identifier for Text (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing off-the-shelf language identification tools favor widely used languages, but Heli-OTS can be used to identify a large group of languages. |
| Approach: | They introduce an off-the-shelf text language identification tool using the HeLI method . they compare the He LI-OTS language identifier with fastText on two different data sets . |
| Outcome: | The proposed language identification tool is compared with fastText on two different data sets. |
Learning to Classify Events from Human Needs Category Descriptions (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that our method outperforms baseline methods, producing substantially better precision. |
| Approach: | They propose a zero-shot learning method that generates weak labels and trains a classifier with weakly labeled data. |
| Outcome: | The proposed method outperforms baseline methods on a human needs categorization task . it produces substantially better precision than baseline methods . |
Evaluating the Knowledge Base Completion Potential of GPT (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Language models (LMs) have been proposed for unsupervised knowledge base completion (KBC) however, their ability to do this at scale and with high accuracy remains an open question. |
| Approach: | They propose to use language models to complete a large public KB, Wikidata, with 90% precision. |
| Outcome: | The proposed models can extend Wikidata by 27M facts at 90% precision. |
Detecting Sexual Content at the Sentence Level in First Millennium Latin Texts (2024.lrec-main)
Copied to clipboard
| Challenge: | a traditional approach to corpus building involves constructing a corpus centered around specific themes, such as colors. |
| Approach: | They propose to use deep learning methods to accelerate corpus building in humanities . they propose to integrate metadata embeddings into the model to improve accuracy . |
| Outcome: | The proposed method outperforms token-based searches in the humanities and linguistics field. |
More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression (2025.findings-emnlp)
Copied to clipboard
Jiebin Zhang, Dawei Zhu, Yifan Song, Wenhao Wu, Chuqiao Kuang, Xiaoguang Li, Lifeng Shang, Qun Liu, Sujian Li
| Challenge: | storing more tokens in the KV cache at lower precision can enhance the long-context performance of large language models. |
| Approach: | They propose a token-precision trade-off strategy to optimize KV cache compression . they also propose storing more tokens in the KV at lower precision . |
| Outcome: | The proposed method achieves an optimal point within the Information Bottleneck compared to standalone KV pruning or KV quantization. |
Downstream Task Performance of BERT Models Pre-Trained Using Automatically De-Identified Clinical Data (2022.lrec-1)
Copied to clipboard
| Challenge: | Automatic de-identification systems introduce errors due to their imperfect precision and may negatively impact the utility of the de-identified dataset. |
| Approach: | They propose to de-identifie a large clinical corpus in Swedish by removing entire sentences containing sensitive data or by replacing sensitive words with realistic surrogates. |
| Outcome: | The proposed models are safe to distribute to other academic researchers and reduce privacy risks. |
BizBench: A Quantitative Reasoning Benchmark for Business and Finance (2024.acl-long)
Copied to clipboard
| Challenge: | Answering questions within business and finance requires reasoning, precision, and a wide-breadth of technical knowledge. |
| Approach: | They propose a benchmark for evaluating models’ ability to reason about realistic financial problems by focusing on question-answering over financial data via program synthesis. |
| Outcome: | The proposed benchmark evaluates models' financial background knowledge, ability to parse financial documents, and capacity to solve complex problems with code. |
Affective Event Classification with Discourse-enhanced Self-training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prior work on recognizing affective events focused on producing lexical resources of verbs or event phrases with corresponding affective polarity values. |
| Approach: | They propose a BERT-based model for affective event classification and a discourse-enhanced self-training method that iteratively improves the classifier with unlabeled data. |
| Outcome: | The proposed model outperforms existing models with unlabeled data and improves recall and precision. |
Multilingual Detection of Personal Employment Status on Twitter (2022.acl-long)
Copied to clipboard
| Challenge: | Detecting disclosures of individuals’ employment status on social media is a challenging task due to their rarity in a sea of social media content and the variety of linguistic forms used to describe them. |
| Approach: | They propose to use BERT-based classification models to identify five types of disclosures about individuals’ employment status in three languages. |
| Outcome: | The proposed methods achieve significant gains in precision, recall, and diversity of results in real-world settings of extreme class imbalance. |
Answer Quality Aware Aggregation for Extractive QA Crowdsourcing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for creating extractive question answering datasets are crowdsourcing, but results are often inconsistent. |
| Approach: | They propose a method for aggregating answers from different crowd workers that takes into account the relations between the answer, question, and context passage. |
| Outcome: | The proposed method outperforms baselines by 16% on precision and effectively conduct answer aggregation for extractive question answering task. |
Towards Verifiable Text Generation with Evolving Memory and Self-Reflection (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) often produce factually incorrect information, also known as hallucination. |
| Approach: | They propose a framework for verifiable text generation with evolving memory and self-reflection that incorporates long-term memory to retain documents and recent documents. |
| Outcome: | The proposed framework outperforms baselines on five datasets across three knowledge-intensive tasks. |
Don’t Annotate, but Validate: a Data-to-Text Method for Capturing Event Data (L18-1)
Copied to clipboard
| Challenge: | Existing methods to create event data are limited by ambiguity and variation in the data. |
| Approach: | They propose a method to obtain large volumes of text corpora with event data . they use a tool to annotate texts and enrich the reference texts with event coreference annotations. |
| Outcome: | The proposed method obtains large volumes of high-quality text corpora with event data . the data obtained with this method have high precision and at a large scale . |
Handling Divergent Reference Texts when Evaluating Table-to-Text Generation (P19-1)
Copied to clipboard
| Challenge: | Existing text generation metrics rely on reference texts, such as BLEU and ROUGE, but they are too expensive to apply repeatedly. |
| Approach: | They propose a metric which aligns n-grams from the generated texts to the semi-structured data before computing their precision and recall. |
| Outcome: | The proposed metric correlates with human judgments better than existing text generation metrics while being easier to use. |
monoQA: Multi-Task Learning of Reranking and Answer Extraction for Open-Retrieval Conversational Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to the Conversational Question Answering task have used multi-task learning to solve the task. |
| Approach: | They propose to use multi-task learning to improve the ORConvQA task by sharing the reranker and reader’s learned structure in a generative model. |
| Outcome: | The proposed model outperforms baseline models on the OR-QuAC and OR-CoQA datasets and significantly outperformed existing strong baseline models. |
Focused-DPO: Enhancing Code Generation Through Focused Preference Optimization on Error-Prone Points (2025.findings-acl)
Copied to clipboard
| Challenge: | Current code generation models produce errors concentrated at specific error-prone points, affecting accuracy of code. |
| Approach: | They propose a framework that focuses preference optimization on error-prone areas . focused-DPO improves the accuracy and reliability of code generation by reducing common errors . |
| Outcome: | The proposed framework improves code generation by focusing on error-prone areas. |
How Universal are Universal Dependencies? Exploiting Syntax for Multilingual Clause-level Sentiment Detection (2020.lrec-1)
Copied to clipboard
| Challenge: | a new method for clause-level sentiment detection is proposed for multilingual use cases. |
| Approach: | They propose a pipeline method that makes the most of syntactic structures based on Universal Dependencies. |
| Outcome: | The proposed method achieves high precision in sentiment detection for 17 languages . it avoids machine-learning approaches that may cause obstacles to its use cases . |
Interpreting Open-Domain Modifiers: Decomposition of Wikipedia Categories into Disambiguated Property-Value Pairs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge repositories rely on Wikipedia for core sets of topics and knowledge assertions. |
| Approach: | They propose an open-domain method for automatically annotating modifier constituents (20th-century’) within Wikipedia categories with properties (date of birth). |
| Outcome: | The proposed method improves precision and recall over a set of Wikipedia categories. |
AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current language models and retrieval-augmented LMs are limited in their ability to perform tasks on the web. |
| Approach: | They propose a benchmark to evaluate language agents built on top of language models . they propose 'AssistantBench' which includes 214 tasks that can be automatically evaluated . |
| Outcome: | The proposed agent outperforms existing agents in a new benchmark for language agents on the web. |
Effects of Human Adversarial and Affable Samples on BERT Generalization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Limiting quantities of training data is considered a key impediment to achieving generalizability in machine learning. |
| Approach: | They examine the impact of training data quality, not quantity, on a model’s generalizability by comparing human-adversarial and human-affable training samples. |
| Outcome: | The proposed model performance improves with 10-30% h-adversarial instances in text classification and relation extraction tasks. |
Detecting Proxy Gaming in RL and LLM Alignment via Evaluator Stress Tests (2026.findings-acl)
Copied to clipboard
| Challenge: | Proxy optimization is a challenge spanning reinforcement learning and LLM alignment. |
| Approach: | They propose an invariance-based framework that detects proxy gaming by separating exploitable sensitivity from content-driven improvements using semantic validity audits. |
| Outcome: | The proposed framework achieves 78.4% precision and 81.7% recall across 15 environments and 5 algorithms. |
Causality-aware Concept Extraction based on Knowledge-guided Prompting (2023.acl-long)
Copied to clipboard
| Challenge: | Concepts in knowledge graphs (KGs) are far from complete in existing knowledge graph models. |
| Approach: | They propose to equip a PLM-based extractor with a knowledge-guided prompt to alleviate concept bias by removing spurious co-occurrence correlations from existing knowledge. |
| Outcome: | The proposed prompt can alleviate concept bias and improve the performance of existing models. |
Towards Evaluation of Cross-document Coreference Resolution Models Using Datasets with Diverse Annotation Schemes (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing cross-document coreference resolution (CDCR) datasets contain event-centric coreference chains of events and entities with identity relations. |
| Approach: | They propose to use a phrasing diversity metric to evaluate lexical diversity of CDCR datasets . they propose to combine CDCR annotation schemes with multiple properties of the coreference chains . |
| Outcome: | The proposed phrasing diversity metric evaluates the CDCR datasets with higher precision. |
Improving Open Information Extraction via Iterative Rank-Aware Learning (P19-1)
Copied to clipboard
| Challenge: | Open information extraction (IE) is the task of extracting open-domain assertions from natural language sentences. |
| Approach: | They propose an additional binary classification loss to calibrate the extraction likelihood . they propose an iterative learning process where extractions generated by the open IE model are incrementally included as training samples to help the model learn from trial and error. |
| Outcome: | Experiments on open information extraction (IE) show that the extraction likelihood is not well calibrated when comparing quality of extracted assertions. |
VulLibGen: Generating Names of Vulnerability-Affected Packages via a Large Language Model (2024.acl-long)
Copied to clipboard
Tianyu Chen, Lin Li, ZhuLiuchuan ZhuLiuchuan, Zongyang Li, Xueqing Liu, Guangtai Liang, Qianxiang Wang, Tao Xie
| Challenge: | Existing work on affected package identification is limited by large language models . a recent study shows that 84% third-party packages contain security vulnerabilities . |
| Approach: | They propose a method to use LLM to generate the affected package . they propose supervised fine-tuning, retrieval augmented generation and a local search algorithm . |
| Outcome: | The proposed method has an average precision of 0.806 for identifying vulnerable packages in four most popular ecosystems in GitHub Advisory. |
Personal Information Leakage Detection in Conversations (2020.emnlp-main)
Copied to clipboard
| Challenge: | 4.5 billion dollars will be invested in conversational assistants (chatbots) by 2021, according to Opus Research 2 . Among diverse types of chatbots, Google Duplex represents the kind of AI personal assistants that act on behalf of people to perform simple tasks. |
| Approach: | They propose to protect personal information by warning users of detected suspicious sentences . they propose to use a constrained alignment problem to perform an alignment optimization problem . |
| Outcome: | The proposed models outperform baseline models on the behavior of personalized chit-chat dialogue systems. |
A Hmong Corpus with Elaborate Expression Annotations (2022.lrec-1)
Copied to clipboard
| Challenge: | SCH is the first substantial corpus to be annotated for elaborate expressions . a plurality of speakers are located in China, but many Hmong speakers left Laos as refugees . |
| Approach: | They describe the first publicly available corpus of Hmong, a minority language of China, Vietnam, Laos, Thailand, and various countries in Europe and the Americas. |
| Outcome: | The first publicly available corpus of Hmong is scraped from a long-running Usenet newsgroup . it is the first substantial corpus to be annotated for elaborate expressions . |
Precise Model Benchmarking with Only a Few Observations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Accurate evaluation of large language models is crucial for identifying their strengths and weaknesses. |
| Approach: | They propose an empirical Bayes estimator that balances direct and regression estimates for each subgroup separately, improving the precision of subgroup-level estimates of model performance. |
| Outcome: | The proposed model reduces the mean squared error by up to 50% on multiple datasets. |
Semantic Evaluation of Multilingual Data-to-Text Generation via NLI Fine-Tuning: Precision, Recall and F1 scores (2025.findings-acl)
Copied to clipboard
| Challenge: | KG-to-Text models are prone to errors like Additions and Omissions, and few languages are taken into account since both train and test data are not readily available. |
| Approach: | They propose a multilingual evaluation framework that is reference-less . it allows estimating how much a KG-to-Text Model under- (omission) or over- (addition) generates. |
| Outcome: | The proposed evaluation framework outperforms prior reference-less metrics in correlation with human judgments and provides scores for precision and recall. |
Grounded Keys-to-Text Generation: Towards Factual Open-Ended Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large pre-trained language models have enabled open-ended generation frameworks to tackle a variety of tasks beyond data-to-text generation. |
| Approach: | They propose a new task to generate a factual description about an entity given guiding keys and grounding passages using a dataset. |
| Outcome: | The proposed model improves factual correctness and recall significantly compared to previous models. |
A Semi-supervised Approach for De-identification of Swedish Clinical Text (2020.lrec-1)
Copied to clipboard
| Challenge: | An abundance of electronic health records (EHRs) is produced every day within healthcare. |
| Approach: | They propose a semi-supervised method for automatically creating high-quality training data for de-identification using annotated data for training and annotations that are costly in time and human resources. |
| Outcome: | The proposed method improves recall from 84.75% to 89.20% without sacrificing precision to the same extent, dropping from 95.73% to 94.20%. |
Class-Adaptive Self-Training for Relation Extraction with Incompletely Annotated Training Data (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing relation extraction models rely on supervised machine learning, but many datasets are incompletely annotated, causing false negatives and errors during inference stage. |
| Approach: | They propose a class-adaptive re-sampling self-training framework that favored the pseudo-labels of classes with high precision and low recall scores. |
| Outcome: | The proposed framework outperforms existing methods on the Re-DocRED and ChemDisgene datasets when the training data are incompletely annotated. |
A Free/Open-Source Morphological Analyser and Generator for Sakha (2022.lrec-1)
Copied to clipboard
| Challenge: | a morphological transducer for Sakha is being developed for use in downstream tasks . the marginalised language is subject to increasing economic and cultural peril due to climate change . |
| Approach: | They describe the development of a morphological analyser and generator for Sakha . the transducer has coverage of solidly above 90%, and high precision . it is already being used in downstream tasks such as linguistic maintenance . |
| Outcome: | The proposed morphological analyser has coverage of 90% and high precision . it is already being used in computer assisted language learning applications . |
AGB-DE: A Corpus for the Automated Legal Assessment of Clauses in German Consumer Contracts (2024.acl-long)
Copied to clipboard
| Challenge: | Annotated contracts are laborious task performed by companies, law firms, NGOs and the scientific community. |
| Approach: | They present a corpus of 3,764 clauses from German consumer contracts annotated by legal experts with a clause in the contract. |
| Outcome: | The proposed framework outperforms openly available models in detecting potentially void clauses. |
A Broad-coverage Corpus for Finnish Named Entity Recognition (2020.lrec-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a fundamental task in natural language processing (NLP). |
| Approach: | They propose to annotate Finnish named entity names using a new corpus built on the Universal Dependencies corpus. |
| Outcome: | The new annotation identifies over 10,000 mentions and maintains compatibility with a previously released single-domain corpus for Finnish NER. |
Evaluating Prompting Strategies for Grammatical Error Correction Based on Language Proficiency (2024.lrec-main)
Copied to clipboard
| Challenge: | generative LLMs have been known for overcorrection where results obtain higher recall measures than precision measures. |
| Approach: | They propose to use generative LLMs to prompt grammatical error correction using a model based on language proficiency to examine the interaction between LLM's performance and L2 language proficiency. |
| Outcome: | The proposed model improves on zero-shot and few-shot prompting and fine-tuning models for grammatical error correction for learners of English as a foreign language based on the different proficiency levels. |
Confront Insider Threat: Precise Anomaly Detection in Behavior Logs Based on LLM Fine-Tuning (2025.coling-main)
Copied to clipboard
| Challenge: | Current methods for insider threat detection suffer from low precision and information loss . a novel approach to detect insider threats is needed to improve accuracy . |
| Approach: | They propose a precise anomaly detection solution based on Large Language Model (LLM) fine-tuning . they represent user behavior in natural language and implement a threat tracing mechanism . |
| Outcome: | The proposed solution achieves an F1 score of 0.8941 on the CERT v6.2 dataset . |
Much Ado About Nothing – Identification of Zero Copulas in Hungarian Using an NMT Model (2020.lrec-1)
Copied to clipboard
| Challenge: | Zero copulas are the phenomenon that nominal predicates lack an explicit verbal copule in default present tense 3rd person indicative cases. |
| Approach: | They propose a tool that can identify and mark the location of zero copulas in Hungarian clauses that contain nominal predicates at the right position. |
| Outcome: | The proposed tool can identify and mark the location of zero copulas, i.e. where an overt copulan would appear in the non-default cases. |
Recommendation Chart of Domains for Cross-Domain Sentiment Analysis: Findings of A 20 Domain Study (2020.lrec-1)
Copied to clipboard
| Challenge: | Cross-domain sentiment analysis (CDSA) is a well-known problem in text analysis, but sufficient datasets may not be available for a domain to be trained. |
| Approach: | They propose to use 11 similarity metrics to facilitate cross-domain sentiment analysis to identify the best domains for CDSA for a given target domain. |
| Outcome: | The proposed approach performs better on 20 domain pairs and is validated by 11 similarity metrics. |
Benchmarking for Domain-Specific LLMs: A Case Study on Academia and Beyond (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Comp-Comp is an iterative benchmarking framework grounded in the principles of comprehensiveness and compactness. |
| Approach: | They propose a benchmark framework that incorporates the principle of comprehensiveness and compactness. |
| Outcome: | The proposed framework is domain-agnostic and adaptable to a wide range of specialized fields. |
Denoising Enhanced Distantly Supervised Ultrafine Entity Typing (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work on distantly supervised (DS) ultra-fine entity typing has received significant attention . however, DS data is noisy and often suffers from missing or wrong labeling issues resulting in low precision and low recall. |
| Approach: | They propose a noise model to estimate unknown labeling noise distribution over input contexts and noisy type labels and a model to train on denoised data. |
| Outcome: | The proposed model outperforms baseline methods on the Ultra-Fine entity typing dataset and OntoNotes dataset. |
Revisiting Supertagging for faster HPSG parsing (2024.emnlp-main)
Copied to clipboard
| Challenge: | a new supertagger for HPSG-based treebanks is used to improve parsing speed and accuracy. |
| Approach: | They propose to integrate the best supertagger into an HPSG-based parser and compare it to an existing system. |
| Outcome: | The proposed system achieves 97.26% accuracy on 950 sentences from WSJ23 and 93.88% on the out-of-domain technical essay The Cathedral and the Bazaar. |
Towards Speaker Verification for Crowdsourced Speech Collections (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing methods to detect low quality work do not address the correctness of the data. |
| Approach: | They propose an unsupervised method for measuring speaker metadata plausibility of a collection, i.e., evaluating the match (or lack thereof) between contributors and speakers. |
| Outcome: | The proposed method shows high precision in automatically classifying contributor alignment (>0.94). |
Finding Spoken Identifications: Using GPT-4 Annotation for an Efficient and Fast Dataset Creation Pipeline (2024.lrec-main)
Copied to clipboard
Maliha Jahan, Helin Wang, Thomas Thebaud, Yinglun Sun, Giang Ha Le, Zsuzsanna Fagyal, Odette Scharenborg, Mark Hasegawa-Johnson, Laureano Moro Velazquez, Najim Dehak
| Challenge: | Existing datasets that are limited to a few dialects, ethnicities, and age groups are not annotated considering these factors. |
| Approach: | They propose a semi-automated dataset creation pipeline that leverages large language models to perform two complex annotation tasks using human annotations as ground truths. |
| Outcome: | The proposed pipeline reduces time required for the filtering and tagging tasks while losing no important information. |
Neural Machine Translation for Mathematical Formulae (2023.acl-long)
Copied to clipboard
| Challenge: | a recent paper examines the problem of neural machine translation of mathematical formulae between ambiguous presentation languages and unambiguous content languages. |
| Approach: | They perform translation tasks from LaTeX to Mathematica and from La TeX into semantic LaTaX using convolutional sequence-to-sequence networks. |
| Outcome: | The proposed translations achieve 95.1% and 90.7% exact matches between the two languages. |
“A Buster Keaton of Linguistics”: First Automated Approaches for the Extraction of Vossian Antonomasia (D19-1)
Copied to clipboard
| Challenge: | Vossian Antonomasia is a stylistic device which attributes a property to a person by naming another person as a reference point. |
| Approach: | They propose a method for the extraction of Vossian Antonomasias that works completely automatically . they use named entity recognition, distant supervision and a bi-directional LSTM . |
| Outcome: | The proposed method outperforms the only existing semi-automatic method for VA identification by more than 30 percentage points in precision. |
ClarQ: A large-scale and diverse dataset for Clarification Question Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing datasets hinder development of large-scale models capable of generating and utilising clarification questions. |
| Approach: | They propose a bootstrapping framework that utilises a neural network architecture to classify clarification questions based on post-comment tuples extracted from stackexchange. |
| Outcome: | The proposed framework aims to increase the accuracy of the classifier and increase recall of clarification questions by applying it to question-answering tasks. |
Identifying Conspiracy Theories News based on Event Relation Graph (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Conspiracy theories are narratives that explains an event or situation in an irrational or malicious manner. |
| Approach: | They propose to integrate an event relation graph into conspiracy theory identification by using soft labels. |
| Outcome: | The proposed approach improves precision and recall of conspiracy theory identification, and generalizes well for new unseen media sources. |
ClaimGen-CN: A Large-scale Chinese Dataset for Legal Claim Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, legal claims are not being used by non-professionals. |
| Approach: | They construct a dataset for Chinese legal claim generation task and then use it to evaluate the generated claims. |
| Outcome: | The proposed dataset is the first for the Chinese legal claim generation task and will be made publicly available. |
BioHopR: A Benchmark for Multi-Hop, Multi-Answer Reasoning in Biomedical Domain (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for multi-hop reasoning in biomedical domain are lacking . bioHopR provides benchmarks to evaluate multi-step reasoning in structured biomedic knowledge graphs . |
| Approach: | They propose a benchmark to evaluate multi-hop, multi-answer reasoning in biomedical knowledge graphs. |
| Outcome: | BioHopR evaluates multi-hop reasoning in biomedical knowledge graphs based on the PrimeKG model . it outperforms proprietary models and open-source biomedal models in 1-hop and 2-hop tasks . |
ActionIE: Action Extraction from Scientific Literature with Programming Languages (2024.acl-long)
Copied to clipboard
Xianrui Zhong, Yufeng Du, Siru Ouyang, Ming Zhong, Tingfeng Luo, Qirong Ho, Hao Peng, Heng Ji, Jiawei Han
| Challenge: | a method that extracts experimental procedures from human language into actionable sequences in robotics language is challenging given the complexity of the instructions and context-dependent nature of the instruction. |
| Approach: | They propose a method that converts actions written in natural language into Python code that can be easily translated into robotics language. |
| Outcome: | The proposed method can extract experimental procedures from human language into actionable sequences in robotics language. |
Unsupervised Domain Clusters in Pretrained Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods to select domains from large corpus of data are often over-simplistic and vague. |
| Approach: | They propose to use pre-trained language models to learn sentence representations that cluster by domains without supervision. |
| Outcome: | The proposed methods outperform established methods on domain selection and precision and recall with respect to an oracle selection. |
Continual Quantization-Aware Pre-Training: When to transition from 16-bit to 1.58-bit pre-training for BitNet language models? (2025.findings-acl)
Copied to clipboard
| Challenge: | Quantization-aware training of large language models reduces the precision of model parameters and reduces memory usage and energy consumption at inference time. |
| Approach: | They propose a method where models are first trained with 16-bit precision and then transition to 1.58-bit quantization-aware training. |
| Outcome: | The proposed training strategy reduces memory and energy consumption while maintaining model accuracy while reducing memory and inference time. |
A Survey of Machine Translation Tasks on Nigerian Languages (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing work on machine translation of low-resource African languages is limited . despite advances in machine translation, there is limited work on Nigerian languages . |
| Approach: | They propose to focus on neural machine translation techniques for Nigerian languages . they outline the limitations of machine translation research on the continent . |
| Outcome: | The proposed research on Nigerian languages highlights the limitations of the current state of the art in machine translation. |
SLANG: New Concept Comprehension of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Dynamic nature of language limits the adaptability of Large Language Models (LLMs) Traditionally, LLMs are trained on static data, which limits their adaptability . |
| Approach: | They propose a benchmark to integrate novel data and assess LLMs’ ability to comprehend emerging concepts, alongside a causal inference-based approach to enhance LLM comprehension of new phrases and their colloquial context. |
| Outcome: | The proposed model outperforms baseline models in terms of precision and relevance in the comprehension of Internet slang and memes. |
Atomic Self-Consistency for Better Long Form Generations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has aimed to improve LLM generations by filtering out hallucinations, thereby improving the accuracy of the information in responses. |
| Approach: | They propose a technique that improves the recall of relevant information in an LLM. |
| Outcome: | The proposed technique improves the recall of relevant information in an LLM. |
Query-Efficient Agentic Graph Extraction Attacks on GraphRAG Systems (2026.acl-long)
Copied to clipboard
| Challenge: | Existing attacks exploit leakage of retrieved subgraphs, leaving the security implications of structured knowledge representations unexplored. |
| Approach: | They propose a framework that leverages a novelty-guided exploration–exploitation strategy and external graph memory modules to extract a latent entity–relation graph. |
| Outcome: | The proposed framework outperforms baselines on medical, agriculture, and literary datasets under identical query budgets while maintaining high precision. |
Boosting Logical Fallacy Reasoning in LLMs via Logical Structure Tree (2024.emnlp-main)
Copied to clipboard
| Challenge: | Logical fallacy is the use of invalid or flawed reasoning in the construction of a statement. |
| Approach: | They propose to build a logical structure tree to represent hierarchical logic flow among relation connectives and their arguments in a statement. |
| Outcome: | The proposed model significantly improves accuracy and recall for fallacy detection and fallacy classification. |
Flowchart-Based Decision Making with Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models face significant challenges in interpretability of dialogue flow and reproducibility of expert knowledge. |
| Approach: | They propose a method that extracts flowcharts from dialogue data and incorporates them into large language models to improve interpretability and reproducibility. |
| Outcome: | The proposed method reconstructs expert decision-making paths with high precision and recall scores on dialogue datasets. |
AutoPKG: An Automated Framework for Dynamic E-commerce Product-Attribute Knowledge Graph Construction (2026.findings-acl)
Copied to clipboard
Pollawat Hongwimol, Haoning Shang, Chutong Wang, Zhichao Wan, Yi Gao, Yuanming Li, Lin Gui, Wenhao Sun, Cheng Yu
| Challenge: | Product attribute extraction in e-commerce is bottlenecked by ontologies that are inconsistent, incomplete, and costly to maintain. |
| Approach: | They propose a multi-agent Large Language Model framework that constructs a Product-attribute Knowledge Graph from multimodal product content. |
| Outcome: | The proposed framework achieves 0.953 WKE for product types, 0.724 WKEs for attribute keys, and 0.531 edge-level accuracy for value assertions after canonicalization on a large real-world marketplace catalog dataset from Lazada (Alibaba). |
Improving Recall of Large Language Models: A Model Collaboration Approach for Relational Triple Extraction (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing large language models can extract triples from simple sentences with few-shot learning or fine-tuning, but they often miss out when extracting from complex sentences. |
| Approach: | They propose an evaluation-filtering framework that integrates large language models with small models for relational triple extraction tasks. |
| Outcome: | The proposed framework integrates large language models with small models for relational triple extraction tasks. |
TV-AfD: An Imperative-Annotated Corpus from The Big Bang Theory and Wikipedia’s Articles for Deletion Discussions (2020.lrec-1)
Copied to clipboard
| Challenge: | Detecting imperatives in oral and written communication is difficult when the user doesn't use the expected forms. |
| Approach: | They created an imperative corpus with dialogues from The Big Bang Theory and Wikipedia comments from Wikipedia . they manually annotated imperatives and used a syntax-based classifier to extract 10,624 statements that may be imperative. |
| Outcome: | The proposed model performs better in the written data compared to speech data, but has a low precision and recall for speech data. |
Improving Autoformalization Using Direct Dependency Retrieval (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for hallucinate formal dependencies lack scalability and precision to leverage ever-growing public datasets. |
| Approach: | They propose a retrieval-augmented framework based on Direct Dependency Retrieval to generate formal dependencies from natural-language mathematical descriptions and verify their existence via an efficient Suffix Array Check (SAC). |
| Outcome: | The proposed framework outperforms state-of-the-art methods in retrieval precision and recall and can be used to validate formal representations in a public dataset. |
Unlocking Legal Knowledge: A Multilingual Dataset for Judicial Summarization in Switzerland (2025.findings-emnlp)
Copied to clipboard
Luca Rolshoven, Vishvaksenan Rasiah, Srinanda Brügger Bose, Sarah Hostettler, Lara Burkhalter, Matthias Stürmer, Joel Niklaus
| Challenge: | a dataset of 20K rulings from the Swiss Federal Supreme Court is lacking in legal headnotes due to the high cost of manual annotation. |
| Approach: | They propose a dataset that contains 20K rulings from the Swiss Federal Supreme Court . they fine-tune open models and compare them to larger general-purpose and reasoning-tunned LLMs . |
| Outcome: | The proposed dataset contains 20K rulings from the Swiss Federal Supreme Court with headnotes in German, French, and Italian. |
QaRL: Rollout-Aligned Quantization-Aware RL for Fast and Stable Training under Training–Inference Mismatch (2026.findings-acl)
Copied to clipboard
Hao Gu, Hao Wang, Jiacheng Liu, Lujun Li, Qiyuan Zhu, Bei Liu, Binxing Xu, Lei Wang, Xintong Yang, Sida Lin, Sirui Han, Yike Guo
| Challenge: | Recent work has shown that reinforcement learning with simple rule-based reward functions (RLVR) can induce emergent reasoning behaviors and yield gains in challenging domains such as math problem solving. |
| Approach: | They propose a rollout-alignment-quantization-aware RL which aligns training-side forward with the quantized rollout to minimize mismatch. |
| Outcome: | The proposed approach outperforms quantized-rollout training by +5.5 on Qwen3-30B-A3B MoE for math problems while maintaining low-bit throughput. |
Faithful and Robust LLM-Driven Theorem Proving for NLI Explanations (2025.acl-long)
Copied to clipboard
| Challenge: | Recent work has shown that the interaction of large language models (LLMs) with theorem provers (TPs) can help verify and improve the validity of NLI explanations. |
| Approach: | They propose to use logical expressions to guide LLMs in generating structured proof sketches and to use them to improve their accuracy. |
| Outcome: | The proposed strategies improve autoformalisation, syntactic errors and explanation refinement over the state-of-the-art model. |
BWLA: Breaking the Barrier of W1AX Post-Training Quantization for LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models have driven major progress in NLP, but memory and compute requirements hinder practical deployment. |
| Approach: | They propose a framework that preserves high accuracy while achieving 1-bit weight quantization . the orthogonal-kronecker transformation learns an orthogonale mapping via EM minimization - a new approach to quantization is proposed . |
| Outcome: | The proposed framework achieves 1-bit weight quantization with low activations with low-bit activations. |
VeriFact: Enhancing Long-Form Factuality Evaluation with Refined Fact Extraction and Reference Facts (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prior work focuses on accuracy and precision, but factuality evaluation is difficult due to inter-sentence dependencies. |
| Approach: | They introduce a factuality evaluation framework to enhance fact extraction . they also introduce 'factRBench' that evaluates both precision and recall . |
| Outcome: | The proposed framework enhances fact extraction by identifying incomplete and missing facts . it also evaluates precision and recall in long-form models, whereas prior work focuses on precision. |
LM-Combiner: A Contextual Rewriting Model for Chinese Grammatical Error Correction (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent work using model ensemble methods based on voting can effectively mitigate over-correction and improve the precision of the GEC system. |
| Approach: | They propose a rewriting model that can directly modify the over-correction of GEC system outputs without a model ensemble. |
| Outcome: | The proposed model can mitigate over-correction and improve accuracy of Chinese grammatical error correction tasks without a model ensemble. |
FactAlign: Long-form Factuality Alignment of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models have demonstrated significant potential as the next-generation information access engines, but reliability is hindered by issues of hallucination and generating non-factual content. |
| Approach: | They propose a novel alignment framework that enhances the factuality of LLMs’ long-form responses while maintaining their helpfulness. |
| Outcome: | The proposed framework improves factuality of LLMs while maintaining helpfulness. |
SedarEval: Automated Evaluation using Self-Adaptive Rubrics (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluation paradigms rely on generic scoring rubrics that fail to consider the specificities of each question and its problem-solving process. |
| Approach: | They propose a new evaluation paradigm based on self-adaptive rubrics that mimic a human evaluator's analytical process. |
| Outcome: | The proposed evaluation paradigm achieves higher concordance rate with human graders than existing paradigms, including GPT-4. |
Morpheme Sense Disambiguation: A New Task Aiming for Understanding the Language at Character Level (2024.lrec-main)
Copied to clipboard
| Challenge: | Morphemes are a strong linguistic feature to capture lexical semantics, but lack of morpheme-informed resources and the expense of manual annotations hinder morphme-enhanced methods. |
| Approach: | They propose a task of Morpheme Sense Disambiguation with two subtasks in-text and in-word to generalize morpheme features on more tasks. |
| Outcome: | The proposed tasks are based on two morpheme-annotated datasets for Chinese . the best model yields a promising precision of 77.66% on in-text and 88.19% on in word . |
Core: Robust Factual Precision with Informative Sub-Claim Identification (2025.findings-acl)
Copied to clipboard
Zhengping Jiang, Jingyu Zhang, Nathaniel Weir, Seth Ebner, Miriam Wanner, Kate Sanders, Daniel Khashabi, Anqi Liu, Benjamin Van Durme
| Challenge: | Using the Decompose-Then-Verify framework, such as FActScore, can be manipulated by adding obvious or repetitive subclaims to artificially inflate scores. |
| Approach: | They propose a decomposition-based tool called Core to filter subclaims based on their uniqueness and informativeness. |
| Outcome: | The proposed evaluation framework supports easy and modular use of Core and various decomposition strategies. |
MULTICOLLAB: A Multimodal Corpus of Dialogues for Analyzing Collaboration and Frustration in Language (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to study complex emotions when a speaker collaborates with a partner are limited. |
| Approach: | They propose to fuse a multimodal dialogue resource with transcribed speech and eye gaze data to create a highly multimodal corpus. |
| Outcome: | The proposed model improves classification accuracy by 21% over baseline using sensor and speech data in 4.5 seconds. |
DiCoRe: Enhancing Zero-shot Event Detection via Divergent-Convergent LLM Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Understanding the complex event ontology, extracting domain-specific triggers from the passage, and structuring them appropriately overloads and limits the utility of Large Language Models (LLMs). |
| Approach: | They propose a divergent-convergent reasoning framework that decouples the task of ED using Dreamer and Grounder. |
| Outcome: | The proposed framework outperforms baselines on six datasets across five domains and nine LLMs, achieving 4–7% average gains over the best baseline. |
Detecting Sockpuppetry on Wikipedia Using Meta-Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to model author-specific sockpuppet detection on Wikipedia are limited in data-scarce settings. |
| Approach: | They propose to use meta-learning to improve model adaptation to a new sockpuppet-group by training models across multiple tasks. |
| Outcome: | The proposed technique improves performance in data-scarce settings by training models across multiple tasks. |
From Prediction to Intervention: Personalized Meal-Level Glucose Regulation via an LLM Agent (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to individualized glucose regulation are generic and do not account for individual-specific glucose dynamics. |
| Approach: | They propose a physio-feedback agentic loop that integrates individualized absorption modeling with dietary intervention to regulate glucose response. |
| Outcome: | The proposed system improves prediction accuracy and reduces glucose excursions. |
TIDES: Technical Information Discovery and Extraction System (2025.emnlp-main)
Copied to clipboard
| Challenge: | Traditional pre-trained LLMs struggle with domain-specific terminology, while fine-tuned LLM requires substantial computational resources. |
| Approach: | They propose a training-free approach that combines TF-IDF with prompt-based LLMs to address technical questions. |
| Outcome: | The proposed system improves the accuracy and efficiency of QA systems in technical domains without LLM retraining. |
Detecting Online Community Practices with Large Language Models: A Case Study of Pro-Ukrainian Publics on Twitter (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for identifying practices within social media are not yet available. |
| Approach: | They propose a methodological workflow for computational identification of such practices within social media texts by using open-source models and OpenAI’s large language models. |
| Outcome: | The proposed method improves accuracy and supports context-sensitive moderation and advancing the understanding of online community dynamics. |
Improving Occupational ISCO Classification of Multilingual Swiss Job Postings with LLM-Refined Training Data (2025.findings-acl)
Copied to clipboard
| Challenge: | 80% of job postings are German, 11% French, 8% English, and under 1% Italian. |
| Approach: | They propose a method that refines silver-standard ISCO labels by consolidating them with predictions from pre-fine-tuned models to resolve discrepancies. |
| Outcome: | The proposed method raises Top-1 accuracy on silver data to 58.3% and reaches 80% precision on held-out data. |
Adaptive Spatial and Temporal Redundancy Optimization for Efficient Reasoning in Large Language Models (2026.acl-long)
Copied to clipboard
Tianle Chen, Pengyu Cheng, Qiyuan Zhu, Jiacheng Wang, Bei Liu, Hao Gu, Ruijie Shen, Xiaofeng Hou, Sirui Han, Jiacheng Liu
| Challenge: | Existing research to improve CoT efficiency falls into three categories, each with distinct limitations. |
| Approach: | They propose a training-free framework that addresses both dimensions of CoT reasoning by applying a progressive precision reduction strategy coupled with an entropy-based confidence mechanism for adaptive termination. |
| Outcome: | Empirical results show that the proposed framework achieves 11.3 efficiency gain without compromising accuracy. |
Reason Only When Needed: Efficient Generative Reward Modeling via Model-Internal Uncertainty (2026.findings-acl)
Copied to clipboard
Chao Xue, Yao Wang, Mengqiao Liu, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Chenyao Lu, Lei Jiang, Yu Lu, Haibo Shi, Shuang Liang, Minlong Peng, Flora D. Salim
| Challenge: | Existing approaches to generating reward models rely on voting-based mechanisms to evaluate CoT outputs. |
| Approach: | They propose an efficient generative reward modeling framework grounded in model-internal uncertainty. |
| Outcome: | The proposed framework reduces inference cost while improving answer accuracy. |
TABARD: A Novel Benchmark for Tabular Anomaly Analysis, Reasoning and Detection (2025.findings-emnlp)
Copied to clipboard
Manan Roy Choudhury, Anirudh Iyengar Kaniyar Narayana Iyengar, Shikhhar Siingh, Sugeeth Puranam, Vivek Gupta
| Challenge: | a new benchmark is constructed to evaluate the accuracy of large language models for tabular data . the benchmark uses direct, indirect, and Chain-of-Thought prompting . |
| Approach: | They propose a framework that uses prompting, self-verification and constraint-based rule execution to improve accuracy. |
| Outcome: | The proposed framework significantly improves accuracy and recall in tabular data. |
Thinking Before Running! Efficient Code Generation with Thorough Exploration and Optimal Refinement (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent research indicates that large language models (LLMs) have demonstrated remark-able capabilities in various programming-related domains, such as code generation and code refinement. |
| Approach: | They propose a framework that combines exploration with refinement to reduce test-time computation overhead. |
| Outcome: | The proposed framework outperforms SOTA and AgentCoder on humanEval and MBPP benchmarks while reducing test-time computation overhead and scalability. |
Entrospect: Information-Theoretic Self-Reflection Elicits Better Response Refinement of Small Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to self-reflection fail to deliver robust response refinement for models with parameter sizes of 10 billion or smaller. |
| Approach: | They propose to redesign Self-Refine and introduce an information-theoretic framework based on Chain-of-Thought prompt engineering to improve self-reflection in Small Language Models. |
| Outcome: | The proposed framework improves reasoning accuracy and computational efficiency by up to 36.2% under identical model and data settings. |
ToMMeR - Efficient Entity Mention Detection from Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to detect text spans that refer to entities are often conflated with entity typing in a single joint task. |
| Approach: | They propose a lightweight model that probes mention detection capabilities from early LLM layers. |
| Outcome: | The proposed model achieves 93% recall zero-shot with 90% precision under human-calibrated LLM-judge protocol . |
The Efficiency vs. Accuracy Trade-off: Optimizing RAG-Enhanced LLM Recommender Systems Using Multi-Head Early Exit (2025.acl-long)
Copied to clipboard
Huixue Zhou, Hengrui Gu, Zaifu Zhan, Xi Liu, Kaixiong Zhou, Yongkang Xiao, Mingfu Liang, Srinivas Prasad Govindan, Piyush Chawla, Jiyan Yang, Xiangfei Meng, Huayu Li, Buyun Zhang, Liang Luo, Wen-Yen Chen, Yiping Han, Bo Long, Rui Zhang, Tianlong Chen
| Challenge: | Existing frameworks for Large Language Models (LLMs) for Click-Through Rate prediction require a careful balance between computational efficiency and predictive accuracy. |
| Approach: | They propose a framework that integrates Retrieval-Augmented Generation with a novel multi-head early exit architecture to address both challenges. |
| Outcome: | The proposed framework reduces retrieval time while maintaining high model performance. |
Should I Trust You? Detecting Deception in Negotiations using Counterfactual RL (2025.findings-acl)
Copied to clipboard
Wichayaporn Wongkamjan, Yanze Wang, Feng Gu, Denis Peskoff, Jonathan K. Kummerfeld, Jonathan May, Jordan Lee Boyd-Graber
| Challenge: | Future human-AI interaction tools can build on our methods for deception detection by triggering friction to give users a chance to interrogate suspicious proposals. |
| Approach: | They propose to use CTRL-D to detect deception in a board game called Diplomacy . CTRL is a counterfactual RL that has a good recall and almost perfect precision . future tools could build on this to reevaluate trust in suspicious negotiations . |
| Outcome: | The proposed method detects human deception with a high precision when compared to a Large Language Model approach that flags many true messages as deceptive. |
SPLICE: A Singleton-Enhanced PipeLIne for Coreference REsolution (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing attempts to integrate singleton mention detection into end-to-end coreference resolution for English have been hampered by the lack of singletont mention spans in the OntoNotes benchmark. |
| Approach: | They propose a two-step neural mention and coreference resolution system that integrates singleton mentions with OntoNotes syntax trees to achieve a near approximation of the Ontonotes dataset with all singletont mentions. |
| Outcome: | The proposed system achieves 94% recall on a sample of gold singletons. |
Toward the Automatic Detection of Word Meaning Negotiation Indicators in Conversation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Word Meaning Negotiations (WMN) are sequences in conversation where speakers collectively discuss and shape word meaning. |
| Approach: | They propose to detect WMN indicators in conversations where a speaker signals the need to clarify or challenge word meaning. |
| Outcome: | The proposed models have better precision than previous regular expression based approaches and show some generalization abilities, but have moderate recall. |
Pre-trained Language Models Learn Remarkably Accurate Representations of Numbers (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing work showed limited success in probing numeric values from models’ representations, indicating that these errors can be attributed to the inherent unreliability of distributionally learned embeddings in representing exact quantities. |
| Approach: | They propose a probing technique that decodes numeric values from input embeddings with near-perfect accuracy across a range of open-source LMs. |
| Outcome: | The proposed probing technique decodes numeric values from input embeddings with near-perfect accuracy across a range of open-source LMs. |
Automated Fine-Grained Mixture-of-Experts Quantization (2025.findings-acl)
Copied to clipboard
| Challenge: | specialized quantization framework for Mixture of Experts architectures is inadequate for model compression. |
| Approach: | They propose a specialized quantization framework for Mixture of Experts architectures . they find that expert networks exhibit distinctive channel-wise outlier distributions ." |
| Outcome: | The proposed framework improves on the Mixtral-8x7b-v0.1 architecture while maintaining minimal computational overhead. |
𝒜3: Automatic Alignment Framework for Attributed Text Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches of aligning large language models to follow user instructions can lead to undue emphasis on irrelevant documents, which in turn reduces the quality of responses. |
| Approach: | They propose to use a framework to automatically generate high-quality attributed query-response pairs for both supervised fine-tuning and preference optimization stages without human annotation. |
| Outcome: | The proposed framework can generate high-quality attributed query-response pairs without human annotation without human intervention. |
Leveraging What’s Overfixed: Post-Correction via LLM Grammatical Error Overcorrection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to reduce overcorrection often result in significantly decreased recall, limiting the usability of correction systems. |
| Approach: | They propose a novel approach that leverages the strengths of large language models to balance recall and precision by triggering overcorrection via LLMs and fine-tuning smaller models to identify and refine erroneous outputs. |
| Outcome: | The proposed approach maximizes recall and precision by leveraging the generative power of LLMs while preserving the reliability of smaller supervised models. |
Unveiling Strengths and Weaknesses of NLP Systems Based on a Rich Evaluation Corpus: The Case of NER in French (2024.lrec-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is an applicative task for which annotation schemes vary . a lack of robustness of some tools towards textual variation limits evaluation . |
| Approach: | They propose a gold corpus for french annotated with a rich tagset that enables comparison with multiple annotation schemes. |
| Outcome: | The proposed framework enables a fair comparison of NER systems across textual genres and annotation schemes. |
Reflective Agreement: Combining Self-Mixture of Agents with a Sequence Tagger for Robust Event Extraction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for event extraction are limited in their ability to recall nuanced or rare events. |
| Approach: | They propose a hybrid approach that leverages a self-mixture of agents and a discriminative sequence tagger to resolve ambiguities and enhance overall event prediction quality. |
| Outcome: | The proposed approach outperforms existing state-of-the-art methods across three benchmark datasets. |
PatentScore: Multi-dimensional Evaluation of LLM-Generated Patent Claims (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing natural language generation (NLG) metrics fail to capture domain-specific nuances . patent claims require precise assessment of structural elements such as antecedent consistency and claim dependency. |
| Approach: | They propose a multi-dimensional evaluation framework specifically designed for patent claims . PatentScore integrates hierarchical decomposition of claim elements, validation patterns and scoring across structural, semantic, and legal dimensions. |
| Outcome: | The proposed evaluation framework outperforms existing evaluation frameworks on patent claims . patentScore achieved highest correlation with expert annotations on 400 patent claims dataset . |
Trident: Self-Supervised Preference Alignment via Triplet Regularization (2026.findings-acl)
Copied to clipboard
| Challenge: | Large vision-Language Models suffer from noisy supervision and semantic ambiguity in self-supervised settings. |
| Approach: | They propose a self-supervised framework that constructs reliable preference triplets . they propose 'trident' objective that enforces semantic separation between the triplet components . |
| Outcome: | The proposed framework outperforms state-of-the-art RLHF and RLAIF benchmarks on LLaVA-1.5-7B and achieves 95.70% precision on POPE using only 4k self-generated triplets and a single epoch. |
FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation (2025.acl-long)
Copied to clipboard
| Challenge: | Language models (LMs) generate false or unverifiable content, often known as hallucination, despite ongoing efforts to enhance their factuality. |
| Approach: | They propose a tool that measures LMs’ factuality in real-world user interactions by evaluating their factual accuracy and categorizing content units as Supported, Unsupported, or Undecidable based on Web-retrieved evidence. |
| Outcome: | The proposed evaluation pipeline measures language models’ factuality in real-world user interactions. |
GAM: Hierarchical Graph-based Agentic Memory for LLM Agents (2026.acl-long)
Copied to clipboard
Zhaofen Wu, Hanrong Zhang, Fulin Lin, Wujiang Xu, Xinran Xu, Yankai Chen, Henry Peng Zou, Shaowen Chen, Weizhi Zhang, Xue Liu, Philip S. Yu, Hongwei Wang
| Challenge: | Current unified stream-based memory systems facilitate context updates but remain vulnerable to interference from transient noise. |
| Approach: | They propose a hierarchical Graph-based Agentic Memory framework that explicitly decouples memory encoding from consolidation to resolve conflict between rapid context perception and stable knowledge retention. |
| Outcome: | The proposed framework outperforms state-of-the-art benchmarks on LoCoMo and LongDialQA. |
3R: Enhancing Sentence Representation Learning via Redundant Representation Reduction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve sentence representations lack fine-grained guidance on reducing redundant information. |
| Approach: | They propose a method that dynamically identifies redundant information from a dimensional perspective and trains the SRL model to redistribute semantics on different dimensions. |
| Outcome: | The proposed method improves sentence representations on seven semantic text similarity benchmarks. |
SUE: Sparsity-based Uncertainty Estimation via Sparse Dictionary Learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to estimate uncertainty use predictive confidence, structural characteristics of representation space, or stochastic variation in model outputs. |
| Approach: | They propose a new uncertainty estimation framework based on sparse dictionary learning by identifying dictionary atoms associated with misclassified samples. |
| Outcome: | The proposed framework outperforms or matches existing methods on several NLU benchmarks and sentiment analysis benchmarks. |
"Excuse me, may I say something..." CoLabScience, A Proactive AI Assistant for Biomedical Discovery and LLM-Expert Collaborations (2026.acl-long)
Copied to clipboard
| Challenge: | Existing large language models (LLMs) are reactive and respond only when prompted, limiting their effectiveness in collaborative settings. |
| Approach: | They introduce a proactive LLM assistant designed to enhance biomedical collaboration between AI systems and human experts through timely, context-aware interventions. |
| Outcome: | The proposed model outperforms baselines in intervention precision and collaborative task utility, highlighting the potential of proactive LLMs as intelligent scientific assistants. |
TopoRAG: Graph-based RAG via Topology-aware Approximate Nearest Neighbor Search (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent studies extend RAG with graph-structured knowledge, enhancing retrieval to capture relational context beyond isolated text chunks. |
| Approach: | They propose a retrieval framework that integrates structural constraints into ANN search . they propose heuristic neighbor expansion which augments the retrieved set by traversing immediate neighbors . |
| Outcome: | The proposed framework improves precision and reduces context redundancy compared to existing methods. |
CSRP: Chain-of-Thought Reasoning for Chinese Text Correction via Reinforcement Learning with Efficiency-Aware Rewards (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models lack specialized priors for subtle grammatical distinctions, and Supervised Fine-Tuning fails to optimize for precision-focused metrics. |
| Approach: | They propose a framework that builds correction capability through Continual Pre-training on 5.9M balanced samples to internalize domain knowledge. |
| Outcome: | The proposed framework outperforms existing models on the NACGEC benchmark with 50.99 F0.5 and 57.17 precision while mitigating over-correction bias. |
The Illusion of Progress: Re-evaluating Hallucination Detection in LLMs (2025.emnlp-main)
Copied to clipboard
Denis Janiak, Jakub Binkowski, Albert Sawczyn, Bogdan Gabrys, Ravid Shwartz-Ziv, Tomasz Jan Kajdanowicz
| Challenge: | Large language models (LLMs) have revolutionized natural language processing, but their tendency to hallucinate poses serious challenges for reliable deployment. |
| Approach: | They propose to use ROUGE to assess lexical overlap to determine accuracy of hallucination detection methods. |
| Outcome: | The proposed evaluation frameworks can rival complex methods, exposing a fundamental flaw in current evaluation practices. |
Introducing Spotlight: A Novel Approach for Generating Captivating Key Information from Documents (2025.emnlp-main)
Copied to clipboard
Ankan Mullick, Sombit Bose, Rounak Saha, Ayan Kumar Bhowmick, Aditya Vempaty, Prasenjit Dey, Ravi Kokku, Pawan Goyal, Niloy Ganguly
| Challenge: | Unlike highlights (fragmented key points) and traditional summaries, spotlights selectively emphasize intriguing content to foster deeper reader engagement with the source material. |
| Approach: | They propose a novel paradigm for information extraction that selectively emphasizes intriguing content to foster deeper reader engagement with the source material. |
| Outcome: | The proposed model improves readability and boosts engagement value of the original document. |
Orthogonal Representation Editing: Decoupling Semantic Entanglement in Batch Knowledge Editing of LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge editing methods suffer from performance degradation in batch knowledge editing. |
| Approach: | They propose an orthogonal representation editing method which decouples semantic entanglement from edit vectors and enforcing orthogonals on edit vector. |
| Outcome: | The proposed method outperforms existing methods and achieves superior performance in cross-lingual knowledge editing scenarios. |
Mechanistic Interpretability of Large-Scale Counting in LLMs through a System-2 Strategy (2026.findings-acl)
Copied to clipboard
Hosein Hasani, Mohammadali Banayeeanzade, Ali Nafisi, Sadegh Mohammadian, Fatemeh Askari, Mobin Bagherian, Amirmohammad Izadi, Mahdieh Soleymani Baghshah
| Challenge: | Large language models exhibit systematic limitations in counting tasks due to depth constraints. |
| Approach: | They propose a method that decomposes large counting tasks into smaller, independent sub-problems that the model can reliably solve. |
| Outcome: | The proposed method surpasses architectural limitations and achieves higher accuracy on large-scale counting tasks. |
Benchmarking Fine-Grained Error Detection in Multimodal Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Multimodal Process Reward Models (MPRMs) have emerged as a pivotal framework for enhancing the reasoning capabilities of Multimodal Large Language Models. |
| Approach: | They propose a benchmark specifically designed to evaluate MPRMs’ proficiency in detecting erroneous reasoning steps across diverse error categories. |
| Outcome: | The proposed model achieves up to 4.8% performance improvement through test-time scaling. |
NSF-SciFy: Mining the NSF Awards Database for Scientific Claims (2026.acl-long)
Copied to clipboard
| Challenge: | NSF-SciFy contains 2.8 million claims from 400,000 abstracts spanning all science and mathematics disciplines. |
| Approach: | They propose to use a dataset to extract scientific claims from National Science Foundation award abstracts and to use it to refine language models. |
| Outcome: | The proposed method improves non-technical abstract generation, claim extraction, and investigation proposal extraction tasks while maintaining high precision and lower recall. |
FactCorrector: A Graph-Inspired Approach to Long-Form Factuality Correction of Large Language Models (2026.acl-long)
Copied to clipboard
Javier Carnerero-Cano, Massimiliano Pronesti, Radu Marinescu, Tigran T. Tchrakian, James Barry, Jasmina Gajcin, Yufang Hou, Alessandra Pascale, Elizabeth M. Daly
| Challenge: | Large language models (LLMs) often produce factually incorrect responses. |
| Approach: | They propose a new method that adapts across domains without retraining and leverages structured feedback to generate a correction. |
| Outcome: | The proposed method outperforms baseline methods on a VELI5 dataset and several popular long-form factuality datasets. |