Papers with production
Constrained Policy Optimization for Controlled Self-Learning in Conversational AI Systems (2023.acl-industry)
Copied to clipboard
| Challenge: | Recent self-learning methods based on user satisfaction metrics and contextual bandits have shown promising results to enable consistent improvements in conversational AI systems. |
| Approach: | They propose a meta-gradient learning approach that adjusts constraint violation penalty terms adaptively through a user-defined meta objective that encourages balanced constraint satisfaction across domains. |
| Outcome: | The proposed framework supports fine-grained exploration targets for individual domains via user-defined constraints. |
Best Practices for Data-Efficient Modeling in NLG:How to Train Production-Ready Neural Models with Less Data (2020.coling-industry)
Copied to clipboard
Ankit Arun, Soumya Batra, Vikas Bhardwaj, Ashwini Challa, Pinar Donmez, Peyman Heidari, Hakan Inan, Shashank Jain, Anuj Kumar, Shawn Mei, Karthik Mohan, Michael White
| Challenge: | Natural language generation (NLG) is a critical component in conversational systems . Traditionally, NLG components have been deployed using template-based solutions . however, deployment of such model-based systems has been challenging due to high latency and data needs. |
| Approach: | They propose a family of techniques to deploy data-efficient neural solutions for NLG in conversational systems to production. |
| Outcome: | The proposed techniques achieve production quality with light-weight neural network models using fraction of the data needed otherwise. |
Domain-specific transformer models for query translation (2023.acl-industry)
Copied to clipboard
| Challenge: | In domains such as Grocery, users prefer to buy certain brands of products . a large non-English speaking population makes it difficult to translate code-mix queries . |
| Approach: | They propose a model to preserve/correct Grocery brand names while translating context words . they propose to use a dataset of popular Groceries brand names to train the model . |
| Outcome: | The proposed model preserves/corrects Grocery brand names while translating context words . it is tested with a large non-English speaking population and is deployed in production . |
Entity resolution for noisy ASR transcripts (D19-3)
Copied to clipboard
| Challenge: | Domain-agnostic Automatic Speech Recognition systems often mistranscribe domain-specific words and phrases. |
| Approach: | They propose a method for handling ASR errors in named entities, specifically person names, for a voice-based collaboration assistant. |
| Outcome: | The proposed method improves accuracy by 40.8% on a voice-based collaboration assistant. |
Label efficient semi-supervised conversational intent classification (2023.acl-industry)
Copied to clipboard
| Challenge: | A conversational chatbot can answer pre-purchase questions and post-purchase queries to provide a seamless shopping experience. |
| Approach: | They propose a semi-supervised learning approach for label-efficient intent classification using a small labeled corpus and large unlabeled query data to train a transformer model. |
| Outcome: | The proposed approach significantly improves over the baseline, even with a limited labeled set. |
Rationale-Guided Distillation for E-Commerce Relevance Classification: Bridging Large Language Models and Lightweight Cross-Encoders (2025.coling-industry)
Copied to clipboard
| Challenge: | Large-scale e-commerce search systems typically follow a multi-step process to retrieve relevant products for a given query. |
| Approach: | They propose a distillation approach that uses "rationales" generated by Large Language Models to guide smaller cross-encoder models. |
| Outcome: | The proposed model achieves ROC-AUC improvements of 1.4% on 9 multilingual e-commerce datasets, 2.4% on 3 ESCI datasets and 6% on GLUE datasets while being 50 times faster per sample. |
Calibrating LLM Judges: Linear Probes for Fast and Reliable Uncertainty Estimation (2026.acl-industry)
Copied to clipboard
Bhaktipriya Radharapu, Eshika Saxena, Kenneth Li, Chenxi Whitehouse, Adina Williams, Nicola Cancedda
| Challenge: | Existing methods for obtaining well-calibrated uncertainty estimates are poorly calibrated or computationally expensive. |
| Approach: | They propose a linear probe that provides calibrated uncertainty estimates from reasoning judges’ hidden states, requiring no additional model training. |
| Outcome: | The proposed method achieves superior calibration compared to existing methods with x computational savings, generalizes robustly to unseen evaluation domains, and delivers higher accuracy on high-confidence predictions. |
Calibrating Imbalanced Classifiers with Focal Loss: An Empirical Study (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Imbalanced data distributions can cause models to overfit to majority classes and output unreliable (mostly overconfident) predictions. |
| Approach: | They propose to streamline the model development and deployment using focal loss to address imbalanced data distributions. |
| Outcome: | The proposed model training with focal loss improves calibration and accuracy compared to standard cross-entropy loss. |
IceBreaker for Conversational Agents: Breaking the First-Message Barrier with Personalized Starters (2026.acl-industry)
Copied to clipboard
Hongwei Zheng, Weiqi Wu, Zhengjia Wang, Guanyu Jiang, Haoming Li, Tianyu Wu, Yongchun Zhu, Jingwu Chen, Feng Zhang
| Challenge: | Existing efforts focus on activation within ongoing dialogues, while overlooking a key real-world bottleneck. |
| Approach: | They propose a conversation starter generation system that generates personalized starters to guide users into conversation without explicit user intent. |
| Outcome: | The proposed system improves user active days by +1.84 and click-through rate by +94.25 and has been deployed in production. |
DeepGen: Diverse Search Ad Generation and Real-Time Customization (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Existing systems that generate ads manually are not effective in generating ad copy and generating millions of ads for large businesses. |
| Approach: | They propose a system that generates fluent ads from advertiser’s web pages in an abstractive fashion and solves practical issues such as factuality and inference speed. |
| Outcome: | The proposed system generates fluent ads from advertiser’s web pages in an abstractive fashion and solves practical issues such as factuality and inference speed. |
Graph-based Multilingual Product Retrieval in E-Commerce Search (2021.naacl-industry)
Copied to clipboard
| Challenge: | Modern e-commerce search systems require product retrieval under multilingual scenarios. |
| Approach: | They propose a universal multilingual retrieval system that captures interactions between search queries and items in e-commerce search. |
| Outcome: | The proposed system outperforms state-of-the-art retrieval models on five countries and has been deployed in production for multiple countries. |
ProCut: LLM Prompt Compression via Attribution Estimation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | ProCut compresses prompts using attribution analysis to reduce prompt size and latency. |
| Approach: | They propose a framework that compresses prompts through attribution analysis using a heuristic and attribution-based attribution model. |
| Outcome: | The proposed framework reduces prompt size by 78% while maintaining or improving task performance by 62%. |
Lattice Path Edit Distance: A Romanization-aware Edit Distance for Extracting Misspelling-Correction Pairs from Japanese Search Query Logs (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods to extract misspelling-correction pairs from Japanese query logs are not effective due to the unique input methods. |
| Approach: | They propose a romanization-aware edit distance that utilizes romanization lattices to efficiently consider all possible romanized forms of input strings. |
| Outcome: | Empirical results show lattice path edit distance outperforms standard edit distance in Japanese . latticae path editing distance outpersforms existing methods even with romanization . |
Cost-effective Deployment of BERT Models in Serverless Environment (2021.naacl-industry)
Copied to clipboard
| Challenge: | a large upfront infrastructure investment makes machine learning models difficult to deploy . however, serverless architectures have strict limits on the size of the deployment package . |
| Approach: | They propose to fine-tune BERT-style models on proprietary datasets for tasks . they use knowledge distillation to obtain models that are tuned for a specific domain . |
| Outcome: | The proposed model deployments report acceptable latency levels and cost-effectiveness without infrastructure overhead. |
Developing Production-Level Conversational Interfaces with Shallow Semantic Parsing (D18-2)
Copied to clipboard
| Challenge: | In this demo, we demonstrate an end-to-end approach for building conversational interfaces from prototype to production. |
| Approach: | They propose an end-to-end approach for building conversational interfaces from prototype to production that leverages shallow semantic parsing. |
| Outcome: | The proposed approach has proven to work well for a number of applications across diverse verticals. |
Hindsight: Structured Agent Memory that Retains, Recalls, and Reflects (2026.acl-demo)
Copied to clipboard
Christopher Latimer, Nicolò Boschi, Andrew Neeser, Chris Bartholomew, Gaurav Srivastava, Xuan Wang, Naren Ramakrishnan
| Challenge: | Hindsight organizes long-term memory into four logical networks and exposes three core operations. |
| Approach: | Hindsight organizes long-term memory into four logical networks and exposes three core operations. |
| Outcome: | Hindsight is a working memory system for AI agents that separates facts from beliefs . the system outperforms existing models on LongMemEval and LoCoMo with 83.6% accuracy . |
CTM - A Model for Large-Scale Multi-View Tweet Topic Classification (2022.naacl-industry)
Copied to clipboard
| Challenge: | Existing methods to classify social media posts into topics have been used to class up documents into topics. |
| Approach: | They propose a neural model that automatically associates social media posts with topics to solve these challenges. |
| Outcome: | The proposed model outperforms existing methods in the context of Twitter where the topic space is 10 times larger with potentially multiple topic associations per Tweet. |
Adaptive Data Flywheel: Applying MAPE Control Loops to AI Agent Improvement (2026.eacl-industry)
Copied to clipboard
Aaditya Shukla, Sidney Knowles, Meenakshi Madugula, David Farris, Ryan Angilly, Santiago Pombo, Lu An, Anbang Xu, Abhinav Balasubramanian, Tan Yu, Jiaxiang Ren, Rama Akkiraju
| Challenge: | NVInfo AI is a generative AI agent that can be deployed in production without full-scale retraining or infrastructure overhauls. |
| Approach: | They propose to implement a retrieval-augmented generation (RAG)-driven data flywheel in NVInfo AI, a mixture-of-experts knowledge assistant, for 30,000 employees. |
| Outcome: | The proposed system addresses failures in retrieval-augmented generation pipelines and enables continuous learning. |
Zshot: An Open-source Framework for Zero-Shot Named Entity Recognition and Relation Extraction (2023.acl-demo)
Copied to clipboard
Gabriele Picco, Marcos Martinez Galindo, Alberto Purpura, Leopold Fuchs, Vanessa Lopez, Thanh Lam Hoang
| Challenge: | ZSL is a machine learning field that uses textual descriptions of entities or relations to perform tasks that are not seen during training. |
| Approach: | They propose a framework that allows researchers to compare state-of-the-art ZSL methods with standard benchmark datasets. |
| Outcome: | The proposed framework compares state-of-the-art methods with benchmark datasets and provides APIs for production under the standard SpaCy NLP pipeline. |
Toward Stronger Textual Attack Detectors (2023.findings-emnlp)
Copied to clipboard
| Challenge: | despite the high performance of deep learning techniques, trained models remain vulnerable to adversarial attacks . authors present LAROUSSE, STAKEOUT and other approaches to detect adversarials . LARousSE is unsupervised, hyperparameter free and non-differentiable . |
| Approach: | They propose a framework to detect adversarial attacks and an extended benchmark to test them . they demonstrate that LAROUSSE outperforms previous methods and allows to identify interesting factor of detection rate variations. |
| Outcome: | The proposed framework outperforms existing methods and allows to identify interesting factor of detection rate variations. |
AUTOSUMM: A Comprehensive Framework for LLM-Based Conversation Summarization (2025.acl-industry)
Copied to clipboard
Abhinav Gupta, Devendra Singh, Greig A Cowan, N Kadhiresan, Siddharth Srivastava, Yagneswaran Sriraja, Yoages Kumar Mantri
| Challenge: | Large language models (LLMs) are used to summarize large volumes of textual information into a smaller, more manageable size. |
| Approach: | They propose a large language model-based summarization system for regulated banking environments that generates accurate, privacy-compliant summaries of customer-advisor conversations. |
| Outcome: | The proposed system achieves 94% factual consistency rate and significant reduction in hallucination rate. |
Grounded Multimodal In-Context Learning for Product Weight Estimation at Scale in E-commerce (2026.acl-industry)
Copied to clipboard
| Challenge: | a large number of e-commerce platforms require manual verification and specialized hardware. |
| Approach: | They propose a multimodal weight estimation framework that uses category-specific exemplars to infer discretized weight buckets. |
| Outcome: | The proposed approach outperforms strong multimodal KNN baselines in accuracy and near-bucket reliability. |
Towards Faithful Industrial RAG: A Reinforced Co-adaptation Framework for Advertising QA (2026.acl-industry)
Copied to clipboard
Wenwei Li, Ming Xu, Tianle Xia, Lingxiang Hu, Yiding Sun, Linfang Shang, Liqun Liu, Peng Shu, Huan Yu, Jie Jiang
| Challenge: | Existing methods for QA in industrial environments are inherently relational and often updated. |
| Approach: | They propose a framework that optimizes retrieval and generation through two components: Graph-aware Retrieval and evidence-constrained reinforcement learning. |
| Outcome: | Experiments on an internal advertising QA dataset show consistent gains across expert-judged dimensions including accuracy, completeness, safety, and URL validity. |
Annotating Research Infrastructure in Scientific Papers: An NLP-driven Approach (2023.acl-industry)
Copied to clipboard
Seyed Amin Tabatabaei, Georgios Cheirmpos, Marius Doornenbal, Alberto Zigoni, Veronique Moore, Georgios Tsatsaronis
| Challenge: | a pipeline is used to identify, extract and link research infrastructure used in scientific publications. |
| Approach: | They propose a natural language processing pipeline for the identification, extraction and linking of Research Infrastructure (RI) used in scientific publications. |
| Outcome: | The proposed pipeline can be used to identify, extract and link research infrastructure used in scientific publications. |
DuIVRS-2: An LLM-based Interactive Voice Response System for Large-scale POI Attribute Acquisition (2026.acl-industry)
Copied to clipboard
| Challenge: | Accurate Point of Interest (POI) attribute acquisition is essential for location-based services, yet traditional IVR systems suffer from error accumulation and high maintenance overhead. |
| Approach: | They propose a large language model-based framework for large-scale POI attribute acquisition at Baidu Maps. |
| Outcome: | The proposed framework outperforms existing IVR systems in 83.9% task success rate while maintaining a low reaction time of 130ms. |
Welcome to the Real World: Efficient, Incremental and Scalable Key Point Analysis (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Key Point Analysis (KPA) extracts the main points from opinions and quantifies their prevalence. |
| Approach: | They propose a key point analysis framework that extracts the main points from opinions and quantifies their prevalence. |
| Outcome: | The proposed system is able to match sentences to key points over five datasets and demonstrate its performance. |
PrismRAG: Boosting RAG Factuality with Distractor Resilience and Strategized Reasoning (2025.emnlp-industry)
Copied to clipboard
Mohammad Kachuee, Teja Gollapudi, Minseok Kim, Yin Huang, Kai Sun, Xiao Yang, Jiaqi Wang, Nirav Shah, Yue Liu, Aaron Colak, Anuj Kumar, Wen-tau Yih, Xin Luna Dong
| Challenge: | Existing methods to improve factuality of large language models (LLMs) rely on human-engineered instructions. |
| Approach: | They propose a retrieval-augmented generation framework that trains the model with distractor-aware QA pairs mixing gold evidence with subtle distractor passages and instills reasoning-centric habits that make the LLM plan, rationalize, and synthesize without extensive human engineered instructions. |
| Outcome: | The proposed framework outperforms state-of-the-art solutions across 12 open-book RAG QA benchmarks and is being deployed in production. |
Advanced Messaging Platform (AMP): Pipeline for Automated Enterprise Email Processing (2025.acl-industry)
Copied to clipboard
Simerjot Kaur, Charese Smiley, Keshav Ramani, Elena Kochkina, Mathieu Sibue, Samuel Mensah, Pietro Totis, Cecilia Tilli, Toyin Aguda, Daniel Borrajo, Manuela Veloso
| Challenge: | a lack of publicly available datasets for training and benchmarking limits current AI techniques' effectiveness in industry-specific applications. |
| Approach: | They propose an email automation pipeline that automates email response generation at scale in real-world enterprise settings. |
| Outcome: | The proposed pipeline automates email response generation at scale in real-world environments. |
Semantic Outlier Removal with Embedding Models and LLMs (2025.acl-industry)
Copied to clipboard
Eren Akbiyik, João F. M. De Almeida, Rik Melis, Ritu Sriram, Viviana Petrescu, Vilhjálmur Vilhjálmsson
| Challenge: | Modern text processing pipelines require robust methods to remove extraneous content while preserving a document’s core message. |
| Approach: | They propose a method that leverages multilingual sentence embeddings and approximate nearest-neighbor search to identify and excise unwanted text segments. |
| Outcome: | Experiments on HTML datasets show that SORE outperforms structural methods and yields high precision in diverse scenarios. |
SLENDER: Structured Outputs for SLM-based NER in Low-Resource Englishes (2025.acl-industry)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) for low-resource variants of English remains challenging, as most models are trained on datasets predominantly focused on American or British English. |
| Approach: | They propose a new output format for Named Entity Recognition (NER) that achieves a three-fold reduction in inference time compared to JSON format. |
| Outcome: | The proposed output format achieves a three-fold reduction in inference time on average compared to JSON format, which is widely used for structured outputs. |
Schema and Natural Language Aware In-Context Learning for Improved GraphQL Query Generation (2025.naacl-industry)
Copied to clipboard
| Challenge: | GraphQL is a flexible alternative to REST APIs, but generating complex queries remains challenging. |
| Approach: | They propose a framework that integrates GraphQL schemas with natural language inputs to improve query generation accuracy. |
| Outcome: | The proposed framework improves performance on a publicly available complex GraphQL dataset. |
Synthesizing and Adapting Error Correction Data for Mobile Large Language Model Applications (2025.acl-industry)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have achieved impressive performance on many language tasks. |
| Approach: | They synthesize a high-quality dataset of error correction pairs to evaluate and improve LLMs for mobile applications by reweighting the sample. |
| Outcome: | The proposed model improves on offline evaluation and live A/B testing, given the LLM performance on offline data and scores from a small privacy-preserving on-device language model. |
MemoPhishAgent: Memory-Augmented Multi-Modal LLM Agent for Phishing URL Detection (2026.acl-industry)
Copied to clipboard
| Challenge: | Traditional phishing website detection relies on static heuristics or reference lists, which lag behind rapidly evolving attacks. |
| Approach: | They propose a memory-augmented multi-modal LLM agent that leverages episodic memories to guide decisions on recurring and novel threats. |
| Outcome: | The proposed agent outperforms state-of-the-art phishing detection tools on two public datasets and improves recall by 20%. |
Query-OPT: Optimizing Inference of Large Language Models via Multi-Query Instructions in Meeting Summarization (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing LLMs require a new call to the inference endpoint/API for each new query . repeated calls to the endpoints/AP Is expensive and impractical for many real-world use cases. |
| Approach: | They compare the performance of various LLMs for query-based meeting summarization . they find that combining queries for the same context in a single prompt can be used to minimize repeated calls. |
| Outcome: | The proposed approach reduces the number of calls to the inference endpoints/APIs in meeting summarization tasks. |
Toward Scalable Verifiable Reward: Proxy State-Based Evaluation for Multi-turn Tool-Calling LLM Agents (2026.acl-industry)
Copied to clipboard
Yun-Shiuan Chuang, Chaitanya Kulkarni, Alec M. Chiu, Avinash Thangali, Zijie Pan, Shivani Shekhar, Yirou Ge, Yixi Li, Uma Kona, Linsey Pang, Prakhar Mehrotra
| Challenge: | Existing agentic benchmarks rely on deterministic backends and are costly to build and iterate. |
| Approach: | They propose a framework that preserves final state-based evaluation without a deterministic database. |
| Outcome: | The proposed framework produces stable, model-differentiating rankings across families and inference-time reasoning efforts. |
TOBUGraph: Knowledge Graph-Based Retrieval for Enhanced LLM Performance Beyond RAG (2025.emnlp-industry)
Copied to clipboard
Savini Kashmira, Jayanaka L. Dantanarayana, Joshua Brodsky, Ashish Mahendra, Yiping Kang, Krisztian Flautner, Lingjia Tang, Jason Mars
| Challenge: | Retrieval-Augmented Generation (RAG) relies on query-chunk text-to-text similarity in the embedding space for retrieval, can fail to capture deeper semantic relationships across chunks, is highly sensitive to chunking strategies, and is prone to hallucinations. |
| Approach: | They propose a graph-based retrieval framework that first constructs the knowledge graph from unstructured data dynamically and automatically. |
| Outcome: | The proposed framework outperforms multiple RAG implementations in both precision and recall, significantly enhancing user experience through improved retrieval accuracy. |
Modelling Language Acquisition through Syntactico-Semantic Pattern Finding (2023.findings-eacl)
Copied to clipboard
| Challenge: | Usage-based theories of language acquisition have documented the processes by which children acquire language through communicative interaction. |
| Approach: | They propose a method for learning grammars based on similarities and differences in linguistic observations alone. |
| Outcome: | The proposed method is able to learn compositional lexical and item-based constructions of variable extent and degree of abstraction, along with a network of emergent syntactic categories. |
Structure-Guided Entity Resolution: Fine-Tuning LLMs for Robust Name Matching in Complex Linguistic Contexts (2026.acl-industry)
Copied to clipboard
| Challenge: | Variations in naming conventions, inconsistent transliteration across scripts, and frequent data entry errors make it difficult to unify user identities, an essential requirement for Know Your Customer (KYC) compliance. |
| Approach: | They propose a framework that fine-tunes an LLM through a two-phase curriculum to match person names across heterogeneous records. |
| Outcome: | The proposed framework outperforms GPT-4o and single-stage fine-tuning baselines in the Indian identity data. |
AutoPenBench: A Vulnerability Testing Benchmark for Generative Agents (2025.emnlp-industry)
Copied to clipboard
Luca Gioacchini, Alexander Delsanto, Idilio Drago, Marco Mellia, Giuseppe Siracusano, Roberto Bifulco
| Challenge: | LLM agents are promising for vulnerability testing, but lack benchmarks to evaluate and compare them. |
| Approach: | They propose an open-source benchmark for the evaluation of vulnerability testing agents that includes 33 tasks ranging from introductory exercises to actual vulnerable systems. |
| Outcome: | The proposed benchmark includes 33 tasks ranging from introductory exercises to actual vulnerable systems. |
Aegis:An Advanced LLM-Based Multi-Agent for Intelligent Functional Safety Engineering (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Aegis is an advanced LLM-based multi-agent for intelligent functional safety engineering that can perform all phases of a vehicle's lifecycle, including design, development, production, operation, and decommissioning. |
| Approach: | They introduce Aegis: An Advanced LLM-Based Multi-Agent for Intelligent Functional Safety Engineering. |
| Outcome: | The proposed solution can perform Hazard Analysis and Risk Assessment (HARA), document Functional Safety Requirements (FSR), and plan test cases for Automatic Emergency Braking (AEB) systems. |
VAULT: VAriable Unified Long Text Representation for Machine Reading Comprehension (2021.acl-short)
Copied to clipboard
| Challenge: | Existing models on Machine Reading Comprehension (MRC) require complex model architecture for effectively modeling long texts with paragraph representation and classification, making inference computationally inefficient for production use. |
| Approach: | They propose a novel Gaussian distribution-based paragraph representation for Machine Reading Comprehension (MRC) that is light-weight and parallel-efficient. |
| Outcome: | The proposed model can achieve comparable performance on Wikipedia-based (NQ) and TechNotes (TechQA) with a state-of-the-art (SOTA) complex document modeling approach while being 16 times faster, demonstrating the efficiency of the proposed model. |
Incremental Summarization for Customer Support via Progressive Note-Taking and Agent Feedback (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Using a mixtral-87B model, we reduce the time it takes to generate concise notes during conversations, and reduce the amount of time spent on manual review. |
| Approach: | They propose a system that combines a Mixtral-87B model for continuous note generation with a DeBERTa-based classifier to filter trivial content. |
| Outcome: | The proposed system achieved a 3% reduction in case handling time compared to bulk summarization, and high agent satisfaction ratings from surveys. |
Learning Selective LLM Autonomy from Copilot Feedback in Enterprise Customer Support Workflows (2026.acl-industry)
Copied to clipboard
Nikita Borovkov, Elisei Rykov, Olga Tsymboi, Sergei Filimonov, Nikita Surnachev, Dmitry Bitman, Anatolii Potapov
| Challenge: | 45% of sessions automated in production without degrading support quality level . traditional automated processes are costly at scale and require manual rule authoring . |
| Approach: | They propose a system that automates end-to-end customer support workflows inside an enterprise BPM platform. |
| Outcome: | The proposed system automates 45% of sessions and reduces average handling time by 39% without degrading support quality level. |
AttributeForge: An Agentic LLM Framework for Automated Product Schema Modeling (2025.emnlp-industry)
Copied to clipboard
Yunhan Huang, Klevis Ramo, Andrea Iovine, Melvin Monteiro, Sedat Gokalp, Arjun Bakshi, Hasan Turalic, Arsh Kumar, Jona Neumeier, Ripley Yates, Rejaul Monir, Simon Hartmann, Tushar Manglik, Mohamed Yakout
| Challenge: | e-commerce platforms are producing only tens of attributes per month for schema modeling . authors present a framework to automate end-to-end product schema modeling using Large Language Models . |
| Approach: | They introduce a framework to automate end-to-end product schema modeling using Large Language Models. |
| Outcome: | The proposed framework achieves an 88 increase in modeling throughput while delivering superior quality. |
Mitigating Hallucinated Translations in Large Language Models with Hallucination-focused Preference Optimization (2025.naacl-long)
Copied to clipboard
| Challenge: | Machine Translation (MT) systems based on fine-tuned large language models (LLMs) are at a higher risk of generating hallucinations, which can severely undermine user’s trust and safety. |
| Approach: | They propose a method that intrinsically learns to mitigate hallucinations during the model training phase. |
| Outcome: | The proposed method reduces hallucinations by 89% on an average across three unseen target languages while preserving translation quality. |
PROMPTEVALS: A Dataset of Assertions and Guardrails for Custom Production Large Language Model Pipelines (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models fail to follow instructions or meet developer expectations when running in production . a dataset of 2087 LLM pipeline prompts with 12623 assertion criteria is larger than previous collections . |
| Approach: | They propose a dataset of 2087 LLM pipeline prompts with 12623 assertion criteria . they fine-tuned Mistral and Llama 3 models outperform GPT-4o by 20.93% on average . |
| Outcome: | The proposed dataset outperforms GPT-4o and mistral models in generating assertions and offers reduced latency and improved performance. |
To Annotate or Not? Predicting Performance Drop under Domain Shift (D19-1)
Copied to clipboard
| Challenge: | Performance drop due to domain-shift is an endemic problem for NLP models in production. |
| Approach: | They propose to use H-divergence, reverse classification accuracy and confidence measures to predict performance drop under domain-shift without any target domain labels. |
| Outcome: | The proposed method predicts performance drops with an error rate as low as 2.15% and 0.89% for sentiment analysis and POS tagging respectively. |
On Continual Model Refinement in Out-of-Distribution Data Streams (2022.acl-long)
Copied to clipboard
| Challenge: | Existing continual learning (CL) problems cannot cover real-world scenarios such as out-of-distribution errors. |
| Approach: | They propose a continual model refinement problem formulation to solve this problem . they extend several existing continual learning approaches to the CMR problem based on a general sampling algorithm . |
| Outcome: | The proposed model refinement solution improves on existing models and their performance metrics. |
Gold Standard Annotations for Preposition and Verb Sense with Semantic Role Labels in Adult-Child Interactions (C18-1)
Copied to clipboard
| Challenge: | Existing corpus of child-directed speech augments existing corpus for semantic role labels . sense and number of arguments were open to multiple interpretations due to rapidly changing discourse . |
| Approach: | They propose to augment an existing corpus of child-directed speech to provide supervised learning of semantic role labels. |
| Outcome: | The resulting corpus is a gold standard for supervised learning of semantic role labels in child-directed speech. |
Building a Word Segmenter for Sanskrit Overnight (L18-1)
Copied to clipboard
| Challenge: | Sanskrit word segmentation is challenging due to the issue of Sandhi . digitisation efforts have made the manuscripts available in the public domain . |
| Approach: | They propose a deep sequence to sequence model that takes only the sandhied string as input and predicts the unsandhized string. |
| Outcome: | The proposed model improves on the current state of the art by 16.79% . the system can be trained "overnight" and be used for production . |
Investigating Productive and Receptive Knowledge: A Profile for Second Language Learning (C18-1)
Copied to clipboard
| Challenge: | Literature on receptive and productive vocabulary often ignores grammar in second language acquisition studies. |
| Approach: | They use two corpora to investigate divergences in grammatical structures in texts . they set a polarity to the divergence scores to indicate whether there is overuse or underuse . |
| Outcome: | The proposed system will help language learners to activate more of their passive knowledge in writing texts. |
Beyond Fair Pay: Ethical Implications of NLP Crowdsourcing (2021.naacl-main)
Copied to clipboard
| Challenge: | Ethical considerations regarding the use of crowdworkers are limited to labor conditions . the Final Rule did not anticipate the use online crowdsourcing platforms for data collection . |
| Approach: | They propose to reopen discussion regarding ethical use of crowdworkers in NLP research . they propose to use online crowdsourcing platforms to evaluate risk of harm . |
| Outcome: | The proposed study identifies common scenarios where crowdworkers performing NLP tasks are at risk of harm. |
ChatGPT Doesn’t Trust Chargers Fans: Guardrail Sensitivity in Context (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing work addresses the limitations of chatbot guardrails, which limit responses to uncertain or sensitive questions. |
| Approach: | They generate user biographies that offer ideological and demographic information about the user. |
| Outcome: | The proposed model can infer a likely political ideology and modify guardrail behavior accordingly. |
The birth of Romanian BERT (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale pretrained language models are available in high-resource languages, in particular English, or as multilingual models that compromise performance on individual languages for coverage. |
| Approach: | They propose to use a Romanian transformer-based language model to pretrained a large text corpus to evaluate the model. |
| Outcome: | The proposed model is open-source and can be used in production. |
Type Enhanced BERT for Correcting NER Errors (2023.findings-acl)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is the task of identifying spans that belong to particular categories, such as person, location, organization, etc. |
| Approach: | They propose a method that integrates named entity’s type information into BERT by an adapter layer and integrates it into a gazetteer. |
| Outcome: | The proposed method outperforms baselines in multiple corpus. |
Few-Shot Document-Level Event Argument Extraction (2023.acl-long)
Copied to clipboard
| Challenge: | Event argument extraction (EAE) has been well studied at the sentence level but under-explored at the document level. |
| Approach: | They propose a Few-Shot Document-Level Event Argument Extraction benchmark to capture event arguments that actually spread across sentences in documents. |
| Outcome: | The proposed task is very challenging with low performance and limited learning process . argument extraction depends on context from multiple sentences and learning process limited to very few examples . |
Another Dead End for Morphological Tags? Perturbed Inputs and Parsing (2023.findings-acl)
Copied to clipboard
| Challenge: | Part-of-speech tags are used for word-contextualized parsers, but their impact is limited to word-based models. |
| Approach: | They propose an adversarial attack to test whether morphological tags contribute to error propagation or correct parsing mistakes. |
| Outcome: | The proposed attack on 14 treebanks shows that if morphological tags were utopically robust against lexical perturbations, they would be able to correct parsing mistakes. |
Endowing Neural Language Learners with Human-like Biases: A Case Study on Dependency Length Minimization (2024.lrec-main)
Copied to clipboard
| Challenge: | Comparing the behavior of models with that of human learners can reveal which aspects affect the emergence of this preference. |
| Approach: | They propose to add three factors to the standard neural-agent language learning and communication framework to make the simulation more realistic. |
| Outcome: | The proposed conditions can contribute to a small but significant learning advantage for listeners of verb-initial languages. |
CantTalkAboutThis: Aligning Language Models to Stay on Topic in Dialogues (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in instruction-tuning datasets focus on specific tasks like mathematical or logical reasoning. |
| Approach: | They propose to use synthetic dialogues to help language models remain focused on the subject at hand during task-oriented interactions. |
| Outcome: | The proposed dataset improves language models' ability to maintain topical coherence compared to general-purpose instruction-tuned LLMs like gpt-4-turbo and Mixtral-Instruct. |
Multi-modal Preference Alignment Remedies Degradation of Visual Instruction Tuning on Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Multi-modal large language models (MLLMs) are expected to support multi-turn queries of interchanging image and text modalities in production. |
| Approach: | They propose to use visual-question-answering (VQA) datasets to annotate a 5k-sample VQA preference dataset and to investigate the degradation of VQA datasets. |
| Outcome: | The proposed model surpasses the instruction-following capabilities of the language model with DPO and SteerLM. |
AutoPKG: An Automated Framework for Dynamic E-commerce Product-Attribute Knowledge Graph Construction (2026.findings-acl)
Copied to clipboard
Pollawat Hongwimol, Haoning Shang, Chutong Wang, Zhichao Wan, Yi Gao, Yuanming Li, Lin Gui, Wenhao Sun, Cheng Yu
| Challenge: | Product attribute extraction in e-commerce is bottlenecked by ontologies that are inconsistent, incomplete, and costly to maintain. |
| Approach: | They propose a multi-agent Large Language Model framework that constructs a Product-attribute Knowledge Graph from multimodal product content. |
| Outcome: | The proposed framework achieves 0.953 WKE for product types, 0.724 WKEs for attribute keys, and 0.531 edge-level accuracy for value assertions after canonicalization on a large real-world marketplace catalog dataset from Lazada (Alibaba). |
Traffic-R1: Reinforced LLMs Bring Human-Like Reasoning to Traffic Signal Control Systems (2026.acl-long)
Copied to clipboard
| Challenge: | Rapid urbanization and surging vehicle ownership intensify congestion . rapid urbanization drives crash rates, slow emergency response, and burden transit-poor communities . |
| Approach: | They introduce a 3B-parameter foundation model with human-like reasoning for Traffic signal control (TSC) they use reinforcement learning and network communication to convert LLM into a traffic-control model that operates like a human traffic agent. |
| Outcome: | The proposed model outperforms baselines and training-intensive RL controllers on a simulated traffic environment and reduces queue lengths by more than 5%. |
Columbo: Expanding Abbreviated Column Names for Tabular Data Using Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing solutions to expand table names are limited by the abbreviated column names of tables. |
| Approach: | They propose to use abbreviated tables to expand column names . they propose to introduce four new datasets with real-world abbrevations . |
| Outcome: | The proposed solution outperforms NameGuess in terms of accuracy and consistency over five datasets. |
CoT-Edit: Reinforcement Learning of Chain-of-Thought Reasoning for Code Edit Suggestion (2026.findings-acl)
Copied to clipboard
| Challenge: | Current code edit models require continuous human guidance to maintain context coherence, thereby disrupting programming flow and increasing cognitive load. |
| Approach: | They propose a reinforcement learning framework that guides LLMs to discover chain-of-thought (CoT) reasoning paths for code editing without requiring human-annotated CoT data. |
| Outcome: | The proposed framework outperforms baselines on an industrial dataset and achieves 60.2% edit accuracy. |
SSSD: Simply-Scalable Speculative Decoding (2026.acl-long)
Copied to clipboard
Michele Marzollo, Jiawei Zhuang, Niklas Roemer, Niklas Zwingenberger, Lorenz K Muller, Lukas Cavigelli
| Challenge: | Existing methods for accelerating inference in Large Language Models require additional training and training, resulting in a higher deployment and maintenance cost. |
| Approach: | They propose a training-free method that combines lightweight n-gram matching with hardware-aware speculation. |
| Outcome: | SSSD reduces latency by up to 2.9 and is faster than autoregressive decoding methods. |
From Where Words Come: Efficient Regularization of Code Tokenizers Through Source Attribution (2026.acl-long)
Copied to clipboard
| Challenge: | Currently, subword tokenization is the most common approach for vocabulary building in large models. |
| Approach: | They propose to regularize training and minimize overfitting by using source-attributed BPE . they find that undertrained tokens are prone to producing unused, unusable tokens . |
| Outcome: | The proposed techniques reduce the number of under-trained tokens while maintaining the same inference procedure as with regular BPE. |
The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Sparse attention is a promising strategy to extend long-context capabilities in LLMs . but its efficiency–accuracy trade-offs remain unclear due to the lack of comprehensive evaluation . |
| Approach: | They evaluate sparse attention methods across multiple model families and sizes . they find larger sparser models outperform smaller dense ones at equivalent cost . |
| Outcome: | The proposed methods outperform smaller sparse models at equivalent cost and improve the Pareto frontier. |
Sign-Language Datasets at Scale: A Comprehensive Survey on Resources, Benchmarks, and Annotation Standards (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks fail to reflect real-world communication needs and are limited in their coverage. |
| Approach: | They present a comprehensive index of sign-language datasets, covering 120 resources across 35 sign languages. |
| Outcome: | The proposed index covers 120 resources across 35 sign languages. |