Papers with large-scale
Selecting Machine-Translated Data for Quick Bootstrapping of a Natural Language Understanding System (N18-3)
Copied to clipboard
| Challenge: | In recent years, there has been growing interest in voice-controlled devices, such as Amazon Alexa or Google home. |
| Approach: | They investigate the use of Machine Translation to bootstrap a natural language understanding system for a new language for the use case of a large-scale voice-controlled device. |
| Outcome: | The proposed method reduces the time and cost of getting annotated corpus for a new language while still providing a large enough coverage of user requests. |
Iterative Stratified Testing and Measurement for Automated Model Updates (2022.emnlp-industry)
Copied to clipboard
Elizabeth Dekeyser, Nicholas Comment, Shermin Pei, Rajat Kumar, Shruti Rai, Fengtao Wu, Lisa Haverty, Kanna Shimizu
| Challenge: | Automating updates to machine learning systems is an important but understudied challenge in AutoML. |
| Approach: | They propose a framework that relies on iterative model building coupled with data-shape stratified model testing and improvement to improve model accuracy. |
| Outcome: | The proposed framework shows a 26% improvement in accuracy for new model use cases on a large-scale NLU system compared to a naive baseline and current cutting-edge methods. |
TTM-RE: Memory-Augmented Document-Level Relation Extraction (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for document-level relation extraction are ineffective in exploiting the full potential of large amounts of training data with varied noise levels. |
| Approach: | They propose a novel approach that integrates a trainable memory module with a noisy-robust loss function that accounts for the positive-unlabeled setting to unlock the full potential of large-scale noisy training data. |
| Outcome: | The proposed model outperforms existing methods on a ReDocRED benchmark dataset with an absolute F1 score improvement of over 3%. |
Self-Aware Feedback-Based Self-Learning in Large-Scale Conversational AI (2022.naacl-industry)
Copied to clipboard
| Challenge: | Large-scale conversational AI systems require constant update to adapt to changing customer behavior and trends . lack of self-awareness in feedback-based systems can cause degradation of performance . et al., e. alderman and scott k. d. argues that such systems are not scalable enough to sustain the rapid update pace of conversational systems. |
| Approach: | They propose a superposition-based model that reactively learns local-adaptive decision boundaries . they propose rewritings with a bi-variate beta setting to improve the model's performance . |
| Outcome: | The proposed model improves the PR-AUC by 27.45% and reduces relative defect reductions by 31.22% . the proposed model can adapt faster to changes in global preferences across a large number of customers . |
Improving Contextual Query Rewrite for Conversational AI Agents through User-preference Feedback Learning (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Contextual query rewriting (CQR) is a crucial component in Conversational AI agents, leveraging contextual information from previous user-agent conversations to improve comprehension of current user intent. |
| Approach: | They propose a framework to enhance the CQR model's capability in generating user preference-aligned rewrites. |
| Outcome: | The proposed framework improves the CQR model's ability to generate user preference-aligned rewrites. |
Error Detection in Large-Scale Natural Language Understanding Systems Using Transformer Models (2021.findings-acl)
Copied to clipboard
| Challenge: | Large-scale conversational assistants process every utterance using multiple models for domain, intent and named entity recognition. |
| Approach: | They combine utterance encodings from a RoBERTa model with the Nbest hypothesis produced by the production system to detect domain classification errors. |
| Outcome: | The proposed approach outperforms bi-LSTM models and a standalone model by 2.2% to 32.2% by ensembling multiple models. |
MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Training (2021.findings-acl)
Copied to clipboard
| Challenge: | Symbolic music understanding is useful for many music applications, but lack of training data hinders representation learning. |
| Approach: | They propose a pre-trained model for music understanding that uses symbolic music data to train music representations. |
| Outcome: | The proposed model improves on four music understanding tasks. |
To Mask or to Mirror: Human-AI Alignment in Collective Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used to model and augment collective decision-making. |
| Approach: | They propose a framework for assessing collective alignment using the Lost at Sea social psychology task. |
| Outcome: | The proposed framework compares LLMs with human-AI alignment on the Lost at Sea social psychology task. |
RoLargeSum: A Large Dialect-Aware Romanian News Dataset for Summary, Headline, and Keyword Generation (2025.coling-main)
Copied to clipboard
Andrei-Marius Avram, Mircea Timpuriu, Andreea Iuga, Vlad-Cristian Matei, Iulian-Marius Taiatu, Tudor Găină, Dumitru-Clementin Cercel, Mihaela-Claudia Cercel, Florin Pop
| Challenge: | Using supervised automatic summarization requires sufficient corpora that include pairs of documents and their summaries. |
| Approach: | They propose a large-scale summarization dataset for the Romanian language that is crawled from publicly available news websites. |
| Outcome: | The proposed system performs well in abstractive summarization, which involves generating new sentences that capture the essence of the original text rather than extracting and rephrasing existing sentences. |
Multi-Granularity Hierarchical Attention Fusion Networks for Reading Comprehension and Question Answering (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to read comprehension style question answering are limited by the volume of annotated datasets. |
| Approach: | They propose a hierarchical attention network for reading comprehension style question answering . they first encode the question and paragraph with fine-grained language embeddings . then propose fusion approach to fuse information from both global and attended representations based on the hierarchic attention network . |
| Outcome: | The proposed method achieves state-of-the-art on the SQuAD and TriviaQA Wiki leaderboards and two adversarial SQu AD datasets. |
Conversational Machine Comprehension: a Literature Review (2020.coling-main)
Copied to clipboard
| Challenge: | Conversational machine comprehension (CMC) is a research track in conversational AI. |
| Approach: | They propose to synthesize a generic framework for CMC models and highlight differences in recent approaches. |
| Outcome: | The proposed model will be used as a compendium for future research. |
MM-SOC: Benchmarking Multimodal Large Language Models in Social Media Platforms (2024.findings-acl)
Copied to clipboard
| Challenge: | Social media platforms are hubs for multimodal information exchange, encompassing text, images, and videos, making it challenging for machines to comprehend the information or emotions associated with interactions in online spaces. |
| Approach: | They propose a benchmark to evaluate MLLMs' understanding of multimodal social media content and a large-scale YouTube tagging dataset to evaluate their performance. |
| Outcome: | The proposed model performs better in a zero-shot setting, suggesting potential improvements. |
A Scalable Framework for Learning From Implicit User Feedback to Improve Natural Language Understanding in Large-Scale Conversational AI Systems (2021.emnlp-main)
Copied to clipboard
Sunghyun Park, Han Li, Ameen Patel, Sidharth Mudgal, Sungjin Lee, Young-Bum Kim, Spyros Matsoukas, Ruhi Sarikaya
| Challenge: | Existing methods to improve NLU are laborintensive and expensive. |
| Approach: | They propose a scalable and automatic approach to improving NLU in a large-scale conversational AI system by leveraging implicit user feedback. |
| Outcome: | The proposed framework improves NLU in a large-scale conversational AI system across 10 domains. |
LLM-REDIAL: A Large-Scale Dataset for Conversational Recommender Systems Created from User Behaviors with LLMs (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing CRS datasets suffer from data inextensibility and semantic inconsistency . |
| Approach: | They introduce the LLM-REDIAL dataset to facilitate the research in CRS by leveraging large language models to generate high-quality dialogues. |
| Outcome: | The proposed dataset is the largest multi-domain CRS dataset which consists of 47.6k multi-turn dialogues with 482.6k utterances across 4 domains. |
mAggretriever: A Simple yet Effective Approach to Zero-Shot Multilingual Dense Retrieval (2023.emnlp-main)
Copied to clipboard
| Challenge: | MLIR requires human annotations in multiple languages, making training labor-intensive. |
| Approach: | They propose a multilingual information retrieval model that leverages pre-trained multilingual transformers for dense retrieval. |
| Outcome: | Empirical results show that mAggretriever outperforms state-of-the-art models fine-tuned on English training data. |
Mitigating Sequential Dependencies: A Survey of Algorithms and Systems for Generation-Refinement Frameworks in Autoregressive Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Sequential dependencies present a fundamental bottleneck in deploying large-scale autoregressive models . |
| Approach: | They analyze methods based on generation strategies and refinement mechanisms . they examine deployment strategies across computing environments and explore applications spanning text, images, and speech generation. |
| Outcome: | The proposed frameworks can be used to improve the quality of autoregressive models. |
Large-Scale Multimodal Knowledge Graph about Classical Chinese Poetry: Fine-grained Method and Comprehensive Evaluation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on classical Chinese poetry are limited by modality constraints, dataset size, or the level of refinement. |
| Approach: | They propose to construct a large-scale and fine-grained multimodal knowledge graph of classical Chinese poetry using an informative ontology graph and a text-image alignment method. |
| Outcome: | The proposed method collects knowledge about classical Chinese poetry from ontology graphs and performs four tasks that demonstrate its comprehensiveness and high quality. |
Themis: A Reference-free NLG Evaluation Language Model with Flexibility and Interpretability (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for evaluation of natural language generation tasks lack reliable data. |
| Approach: | They propose to use annotations from human and GPT-4 to construct a corpus for NLG evaluation. |
| Outcome: | The proposed corpus can perform flexible and interpretable evaluations without references and surpasses existing models. |
Robust Utility-Preserving Text Anonymization Based on Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing techniques face challenges of re-identification ability of large language models . anonymizing text that contains sensitive information is crucial for a wide range of applications . |
| Approach: | They propose a framework that integrates three key LLM components to perform anonymization. |
| Outcome: | The proposed model outperforms baselines while maintaining greater data utility in downstream tasks. |
LLM-FK: Multi-Agent LLM Reasoning for Foreign Key Detection in Large-Scale Complex Databases (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for detecting missing foreign keys are limited in capturing semantic dependencies across schemas. |
| Approach: | They propose a framework that integrates four agents to detect missing foreign keys . they propose combinatorial search space explosion, ambiguous inference and global inconsistency . |
| Outcome: | The proposed framework achieves F1-scores above 93% on large-scale MusicBrainz database . it reduces candidate search space by two to three orders of magnitude without losing true FKs . |