Papers with speed
Learning LLM Preference over Intra-Dialogue Pairs: A Framework for Utterance-level Understandings (2025.naacl-industry)
Copied to clipboard
Xuanqing Liu, Luyang Kong, Wei Niu, Afshin Khashei, Belinda Zeng, Steve Johnson, Jon Jay, Davor Golac, Matt Pope
| Challenge: | Large language models (LLMs) have demonstrated remarkable capabilities in handling complex dialogue tasks without requiring use case-specific fine-tuning. |
| Approach: | They propose a framework that combines the scalability of LLM-generated labels with the precision of human annotations to achieve higher speed and accuracy comparable to larger models. |
| Outcome: | The proposed framework significantly improves accuracy across utterance-level dialogue tasks, including sentiment detection (over 2%), dialogue act classification (over 1.5%), etc. |
Guided K-best Selection for Semantic Parsing Annotation (2022.acl-demo)
Copied to clipboard
Anton Belyy, Chieh-yang Huang, Jacob Andreas, Emmanouil Antonios Platanios, Sam Thomson, Richard Shin, Subhro Roy, Aleksandr Nisnevich, Charles Chen, Benjamin Van Durme
| Challenge: | a prototype model trained on a small amount of data is not available, leading to limited prediction performance. |
| Approach: | They propose a human-in-the-loop process that generates a set of valid candidates and allows users to quickly traverse the set and filter incorrect parses. |
| Outcome: | The proposed process can be used to efficiently traverse the candidate set and select the correct parse, with minimal modification when necessary. |
Pieces of Eight: 8-bit Neural Machine Translation (N18-3)
Copied to clipboard
| Challenge: | Neural machine translation models are trained using 32-bit floating point values and have improved fluency and adequacy. |
| Approach: | They propose to use 8-bit quantization to train models using 32-bit floating point values and show that 8- bit translation makes a non-negligible impact in terms of speed with no degradation in accuracy and adequacy. |
| Outcome: | The proposed method improves accuracy and accuracy without degradation in accuracy and adequacy. |
Token Level Routing Inference System for Edge Devices (2025.acl-demo)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been gaining in performance but deployment in edge devices faces significant hurdles due to their high computational complexity. |
| Approach: | They propose a collaborative decoding system that allows small models to perform on-device inference while selectively consulting a cloud-based large model for critical token generation. |
| Outcome: | The proposed system achieves 60% performance gain on CommonsenseQA using a 0.5B model on an M1 MacBook, with under 7% of tokens generation uploaded to the large model in the cloud. |
FastFit: Fast and Effective Few-Shot Text Classification with a Multitude of Classes (2024.naacl-demo)
Copied to clipboard
| Challenge: | Few-shot prompting of large language models (LLMs) via API calls presents a unique challenge when dealing with a multitude of classes that share similar semantic meanings. |
| Approach: | They present a Python package that integrates batch contrastive learning and token-level similarity score to provide fast few-shot classification. |
| Outcome: | The proposed method significantly improves multi-class classification speed and accuracy across English and Multilingual datasets. |
LIDA: Lightweight Interactive Dialogue Annotator (D19-3)
Copied to clipboard
| Challenge: | Dialogue systems are dependent on the quality of the data used to train them. |
| Approach: | They propose to develop an annotation tool specifically for conversation data that handles the entire dialogue annotation pipeline from raw text to structured conversation data. |
| Outcome: | The proposed tool handles the entire dialogue annotation pipeline from raw text to structured conversation data and has a dedicated interface to resolve inter-annotator disagreements. |
Lightweight Transformers for Conversational AI (2022.naacl-industry)
Copied to clipboard
| Challenge: | Commercial dialogue systems typically require a small footprint and fast execution time, but recent trends are in the other direction, resulting in difficulties in model deployment. |
| Approach: | They build Transformer-based Language Models from scratch on large corpora of conversational data and compare their performance against BERT and other strong baselines on dialogue probing tasks. |
| Outcome: | The proposed model outperforms existing models on dialogue probing tasks and can be fine-tuned on a single consumer GPU card. |
Application-Agnostic Language Modeling for On-Device ASR (2023.acl-industry)
Copied to clipboard
| Challenge: | On-device automatic speech recognition systems face several challenges compared to server-based systems. |
| Approach: | They propose to use a feed-forward language model to build a single application-agnostic model . they propose to reduce disk size by half while maintaining speed and accuracy of original model a . |
| Outcome: | The proposed architecture reduces disk size by half while maintaining speed and accuracy of the original model. |
A Fast, Compact, Accurate Model for Language Identification of Codemixed Text (D18-1)
Copied to clipboard
| Challenge: | a feed-forward network can label codemixed and monolingual text in 100 languages and 100 language pairs. |
| Approach: | They propose a feed-forward network that can provide a language code for every token in a sentence . they show that the model can label both codemixed and monolingual text in 100 languages . |
| Outcome: | The proposed model outperforms previous multilingual approaches in terms of accuracy and speed. |
A Cross-lingual Messenger with Keyword Searchable Phrases for the Travel Domain (C18-2)
Copied to clipboard
| Challenge: | Query Translator is a cross-lingual messaging app for the travel domain that automatically translates conversations . the application addresses common cross-linguistic communication issues such as translation accuracy, speed, privacy and personalization. |
| Approach: | They present a cross-lingual messaging app that automatically translates conversations while supporting keyword-to-sentence matching. |
| Outcome: | The proposed app translates conversations while supporting keyword-to-sentence matching. |
SaFER: A Robust and Efficient Framework for Fine-tuning BERT-based Classifier with Noisy Labels (2023.acl-industry)
Copied to clipboard
| Challenge: | Existing noise-handling methods could not improve performance of BERT on noisy datasets . existing methods could only improve performance on noisy data, authors say . |
| Approach: | They propose a fine-tuning framework for BERT-based text classifiers that combats label noises without access to clean data for training or validation. |
| Outcome: | The proposed framework achieves superior performance on multiple text classification benchmarks. |
TestAgent: An Adaptive and Intelligent Expert for Human Assessment (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing adaptive testing methods face several challenges due to mechanized nature of most algorithms and noisy response data. |
| Approach: | They propose to use large language models to enhance adaptive testing through interactive engagement to capture test-takers’ responses and anomalies. |
| Outcome: | The proposed agent achieves more accurate results with 20% fewer questions than state-of-the-art baselines and testers preferred it in speed, smoothness, and other dimensions. |
FAST: Fast Annotation tool for SmarT devices (2021.emnlp-demo)
Copied to clipboard
| Challenge: | In real-world applications, annotators with the same attributes are required to annotate whether the outputs of natural language generation systems are fluent or not. |
| Approach: | They propose an annotation tool for application tasks that focuses on the user experience of mobile devices and can be customized to fit various tasks. |
| Outcome: | The proposed tool can annotate faster than existing methods while maintaining the quality of annotation. |
Speed Without Sacrifice: Fine-Tuning Language Models with Medusa and Knowledge Distillation in Travel Applications (2025.acl-industry)
Copied to clipboard
Daniel Zagyva, Emmanouil Stergiadis, Laurens Van Der Maas, Aleksandra Dokic, Eran Fainman, Ilya Gusev, Moran Beladev
| Challenge: | Rapid growth of digital applications has intensified the demand for real-time natural language processing (NLP) capabilities. |
| Approach: | They propose a framework that combines Medusa and knowledge distillation to achieve compounded benefits in both model size and inference speed. |
| Outcome: | The proposed framework reduces inference latency by 10-20x while maintaining the student model’s performance quality. |
PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them (2021.tacl-1)
Copied to clipboard
Patrick Lewis, Yuxiang Wu, Linqing Liu, Pasquale Minervini, Heinrich Küttler, Aleksandra Piktus, Pontus Stenetorp, Sebastian Riedel
| Challenge: | Open-domain Question Answering models that directly leverage question-answer (QA) pairs show promise in terms of speed and memory compared with conventional models which retrieve and read from text corpora. |
| Approach: | They propose a question-answer (QA)-pair retriever to facilitate improved QA-patch models by introducing Probably Asked Questions (PAQ) they propose QA pair retriever, RePAQ, which preempts and caches test questions, enabling it to match the accuracy of recent retrieve-and-read models, whilst being significantly faster. |
| Outcome: | The proposed model outperforms baseline models by 5% but trails RePAQ by 15% . it can be configured for size (under 500MB) or speed (over 1K questions per second) while retaining high accuracy. |
LLMs on a Budget? Say HOLA (2025.emnlp-industry)
Copied to clipboard
Zohaib Hasan Siddiqui, Jiechao Gao, Ebad Shabbir, Mohammad Anas Azeez, Rafiq Ali, Gautam Siddharth Kashyap, Usman Naseem
| Challenge: | Current solutions such as quantization, pruning, and Retrieval-Augmented Generation (RAG) offer only partial optimizations and often sacrifice accuracy, speed, or generality. |
| Approach: | They propose an end-to-end optimization framework for efficient LLM deployment . it leverages Hierarchical Speculative Decoding (HSD) for faster inference without quality loss. |
| Outcome: | HOLA delivers +17.6% EMA on GSM8K, +10.5% MCA on ARC, and reduced latency and memory on edge devices like Jetson Nano. |
RAGulator: Lightweight Out-of-Context Detectors for Grounded Text Generation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | In enterprise settings, Generative AI has received widespread adoption as a tool to uplift employees' productivity. |
| Approach: | They develop lightweight models capable of detecting when LLM-generated text deviates from retrieved source documents semantically. |
| Outcome: | The proposed models outperform open-source alternatives on credit policy and sustainability reports used in the banking industry. |
CEV-LM: Controlled Edit Vector Language Model for Shaping Natural Language Generations (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing control approaches primarily adjust the semantic (e.g., emotion, topics), structural (e-speech, parts-of-seech), and lexical (el-s-sp-s) properties of text, but are insufficient to accomplish complex objectives such as pacing which control the complexity and readability of the text. |
| Approach: | They propose a lightweight semi-autoregressive language model that uses edit vectors to control three complementary metrics that quantify the shape of text. |
| Outcome: | The proposed model provides significantly more targeted and precise control of speed, volume, and circuitousness while using less training data, and containing fewer parameters. |
Parsing Tweets into Universal Dependencies (N18-1)
Copied to clipboard
| Challenge: | a new tweet treebank for English is designed to analyze tweets with universal dependencies (UD). |
| Approach: | They extend the universal dependencies guidelines to include special constructions in tweets that affect tokenization, part-of-speech tagging, and labeled dependencies. |
| Outcome: | The proposed method outperforms state-of-the-art parsers on other treebanks in accuracy and speed. |
k-SemStamp: A Clustering-Based Semantic Watermark for Detection of Machine-Generated Text (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent watermarked generation algorithms inject detectable signatures during language generation to facilitate post-hoc detection. |
| Approach: | They propose a watermark which assigns signatures to each watermarked sentence according to locality-sensitive hashing (LSH) they propose k-SemStamp, which uses kmeans clustering to partition the semantic space with awareness of inherent semantic structure. |
| Outcome: | The proposed watermark improves its robustness and sampling efficiency while preserving the generation quality, making it more effective for machine-generated text detection. |
Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systems (2025.emnlp-industry)
Copied to clipboard
Kayhan Behdin, Ata Fatahibaarzi, Qingquan Song, Yun Dai, Aman Gupta, Zhipeng Wang, Hejian Sang, Shao Tang, Gregory Dexter, Sirou Zhu, Siyu Zhu, Tejas Dharamsi, Vignesh Kothapalli, Zhoutong Fu, Yihan Cao, Pin-Lun Hsu, Fedor Borisyuk, Natesh S. Pillai, Luke Simon, Rahul Mazumder
| Challenge: | Large language models (LLMs) have demonstrated remarkable performance across a wide range of industrial applications. |
| Approach: | They propose two techniques for training and deploying small language models that deliver high performance for a variety of industry use cases. |
| Outcome: | The proposed techniques retain much of the quality of larger models while reducing training/serving costs and latency. |
Guiding Neural Machine Translation with Retrieved Translation Pieces (N18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has trouble with lowfrequency words or phrases and generalizing across domains. |
| Approach: | They propose a method for recalling low-frequency words and phrases into neural machine translation by retrieving n-grams from a search engine and incorporating them into the decoding process. |
| Outcome: | The proposed method improves translation results up to 6 BLEU points on three narrow domain translation tasks where repetitiveness of the target sentences is particularly salient. |
KTRL+F: Knowledge-Augmented In-Document Search (2024.naacl-long)
Copied to clipboard
| Challenge: | KTRL+F is a knowledge-augmented in-document search that requires real-time identification of all semantic targets within a document with the awareness of external sources through a single natural query. |
| Approach: | They propose a knowledge-augmented in-document search that requires real-time identification of all semantic targets within a document with the awareness of external sources through a single natural query. |
| Outcome: | The proposed model reduces time for searching with less queries and reduced extra visits to other sources for collecting evidence. |
AttributeForge: An Agentic LLM Framework for Automated Product Schema Modeling (2025.emnlp-industry)
Copied to clipboard
Yunhan Huang, Klevis Ramo, Andrea Iovine, Melvin Monteiro, Sedat Gokalp, Arjun Bakshi, Hasan Turalic, Arsh Kumar, Jona Neumeier, Ripley Yates, Rejaul Monir, Simon Hartmann, Tushar Manglik, Mohamed Yakout
| Challenge: | e-commerce platforms are producing only tens of attributes per month for schema modeling . authors present a framework to automate end-to-end product schema modeling using Large Language Models . |
| Approach: | They introduce a framework to automate end-to-end product schema modeling using Large Language Models. |
| Outcome: | The proposed framework achieves an 88 increase in modeling throughput while delivering superior quality. |
WASA: A Web Application for Sequence Annotation (L18-1)
Copied to clipboard
| Challenge: | a major barrier to research on CS has been the lack of large multilingual, multi-genre CS-annotated corpora. |
| Approach: | They propose a web-based annotation system that manages large-scale CS data annotation. |
| Outcome: | The proposed system can manage large-scale multilingual code switching (CS) data annotation. |
Beyond Benchmarks: Building a Richer Cross-Document Event Coreference Dataset with Decontextualization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing datasets for Cross-Document Event Coreference (CDEC) are small and lacking diversity. |
| Approach: | They propose a new approach leveraging large language models to decontextualize event mentions by simplifying the document-level annotation task to sentence pairs with enriched context. |
| Outcome: | The proposed approach improves the quality of the dataset and generalizability of the model. |
Dissecting Contextual Word Embeddings: Architecture and Representation (D18-1)
Copied to clipboard
| Challenge: | Existing work on learning contextual representations has used LSTM-based biLMs, but there is no reason to believe this is effective. |
| Approach: | They propose to use pre-trained bidirectional language models to learn contextual word embeddings for four NLP tasks and to use them to study the effects of architecture on endtask accuracy. |
| Outcome: | The proposed models outperform word embeddings for four NLP tasks and all learn representations that vary with network depth. |
Build Fast and Accurate Lemmatization for Arabic (L18-1)
Copied to clipboard
| Challenge: | Lemmatization is the process of finding the base form (lemma) of a word by considering its inflected forms. |
| Approach: | They propose a lemmatizer for Arabic with a dataset that can be used to test lemma accuracy. |
| Outcome: | The proposed algorithm outperforms state-of-the-art Arabic lemmatization in accuracy and speed. |
EMR Coding with Semi-Parametric Multi-Head Matching Networks (N18-1)
Copied to clipboard
| Challenge: | Electronic medical record (EMR) coding is the process of extracting diagnosis and procedure codes from the digital record (the EMR) pertaining to a patient's visit. |
| Approach: | They propose a neural network architecture that combines ideas from few-shot learning matching networks, multi-label loss functions, and convolutional neural networks for text classification to significantly outperform other state-of-the-art models. |
| Outcome: | The proposed model outperforms existing models on a well known de-identified EMR dataset with multi-label performance measures. |
LLM4DistReconfig: A Fine-tuned Large Language Model for Power Distribution Network Reconfiguration (2025.naacl-long)
Copied to clipboard
| Challenge: | Power distribution network reconfiguration is crucial for maintaining operational efficiency, reliability and adaptability in modern power networks. |
| Approach: | They propose a deep learning-based approach to solve a distribution network reconfiguration problem using inputs from a LLM. |
| Outcome: | The proposed model generates optimal configurations minimizing system loss for five individual and a combined test dataset. |
Joint Embedding of Words and Labels for Text Classification (P18-1)
Copied to clipboard
Guoyin Wang, Chunyuan Li, Wenlin Wang, Yizhe Zhang, Dinghan Shen, Xinyuan Zhang, Ricardo Henao, Lawrence Carin
| Challenge: | Existing approaches to text classification use word embeddings to capture semantic regularities between words. |
| Approach: | They propose to view text classification as a label-word joint embedding problem . they use a framework that measures compatibility between text sequences and labels . |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on large text datasets. |
Semi-Supervised Dependency Parsing with Arc-Factored Variational Autoencoding (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for dependency parsing use unlabelled data to compensate for the lack of training corpora. |
| Approach: | They propose semi-supervised dependency parsing methods that utilize unlabelled data to compensate for the scarcity of labelled training corpora. |
| Outcome: | The proposed model overcomes the tree constraint and the complexity of the training procedure while avoiding the challenges brought by the tree constraints. |
Revisiting Early Detection of Sexual Predators via Turn-level Optimization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to detect online grooming rely on chat-level risk labels and fail to identify optimal intervention points. |
| Approach: | They propose a speed control reinforcement learning strategy based on luring communication theory to capture the predator’s turn-level entrapment and a new reward function that balances the trade-off between speed and accuracy based upon the LCT. |
| Outcome: | The proposed method preempts online grooming while identifying optimal early intervention points. |
Benchmarking Approximate Inference Methods for Neural Structured Prediction (N19-1)
Copied to clipboard
| Challenge: | Structured prediction models often involve complex inference problems for which finding exact solutions is intractable. |
| Approach: | They propose to perform gradient descent with respect to the output structure directly and train a neural network to perform inference. |
| Outcome: | The proposed methods achieve better speed/accuracy/search error trade-off than gradient descent while being faster than exact inference at similar accuracy levels. |
Dynamic Feature Fusion for Sign Language Translation Using HyperNetworks (2025.findings-naacl)
Copied to clipboard
| Challenge: | Using RGB and keypoint streams, sign language translation is highly dependent on the brain's ability to process color, shape, and motion simultaneously. |
| Approach: | They propose a hypernetwork-based fusion method that extracts salient features from RGB and keypoint streams and introduces self-distillation and SST contrastive learning to maintain feature advantages while aligning the global semantic space. |
| Outcome: | The proposed method achieves state-of-the-art performance on two public sign language datasets, reducing model parameters by about two-thirds. |
LittleBird: Efficient Faster & Longer Transformer for Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for BERT have a limitation dealing with long inputs due to its attention mechanism. |
| Approach: | They propose a model based on BigBird with improved speed and memory footprint . they propose 'pack and unpack attention' to replace global attention . |
| Outcome: | The proposed model can work on long inputs even after being pre-trained on short inputs. |
Enriched In-Order Linearization for Faster Sequence-to-Sequence Constituent Parsing (2020.acl-main)
Copied to clipboard
| Challenge: | Sequence-to-sequence constituent parsing requires a linearization to represent trees as sequences. Top-down tree linearizations have achieved the best accuracy to date. |
| Approach: | They propose to use an in-order shift-reduce linearization instead of a top-down tree linearization to represent trees as sequences. |
| Outcome: | The proposed approach achieves the best accuracy to date on the English PTB dataset among fully-supervised single-model sequence-to-sequence constituent parsers. |
Syllable-based Neural Thai Word Segmentation (2020.coling-main)
Copied to clipboard
| Challenge: | Word segmentation is a challenging pre-processing step for Thai Natural Language Processing due to the lack of explicit word boundaries. |
| Approach: | They propose a neural Thai word segmenter that uses syllable embeddings to capture linguistic constraints and dilated CNN filters to capture the environment of each character. |
| Outcome: | The proposed system outperforms the previous state-of-the-art system on both in-domain and out-domain datasets. |
Attend, Select and Eliminate: Accelerating Multi-turn Response Selection with Dual-attention-based Content Elimination (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models can be used to perform multi-turn response selection, but they can be expensive. |
| Approach: | They propose a framework and a strategy that progressively selects and eliminates unimportant content under context-response dual-attention. |
| Outcome: | The proposed method can effectively speed-up SOTA models without much performance degradation and shows a better trade-off between speed and performance than previous methods. |
AnchorAttention: Difference-Aware Sparse Attention with Stripe Granularity (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for large language models with extended context lengths face significant computational challenges during the prefill phase. |
| Approach: | They propose a difference-aware, dynamic sparse attention mechanism that efficiently identifies critical attention regions at a finer stripe granularity while adapting to global contextual information. |
| Outcome: | The proposed model achieves a speedup of 1.44 while maintaining higher recall rates. |
FastDiff 2: Revisiting and Incorporating GANs and Diffusion Models in High-Fidelity Speech Synthesis (2023.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that Generative adversarial networks sacrifice sample diversity for quality and speed, while diffusion models exhibit outperformed sample quality and diversity at a high computational cost. |
| Approach: | They propose to combine GANs and diffusion probabilistic models to achieve better sample quality and diversity. |
| Outcome: | The proposed models outperform GANs and diffusion models in speech synthesis . the proposed models enjoy an efficient 4-step sampling process and exhibit better sample diversity . |
Bitnet.cpp: Efficient Edge Inference for Ternary LLMs (2025.acl-long)
Copied to clipboard
Jinheng Wang, Hansong Zhou, Ting Song, Shijie Cao, Yan Xia, Ting Cao, Jianyu Wei, Shuming Ma, Hongyu Wang, Furu Wei
| Challenge: | 1-bit large language models have spurred interest in ternary LLMs, but efficient edge inference is still scarce. |
| Approach: | They propose an inference system optimized for 1-bit large language models . they propose a new library that facilitates sub-2-bits-per-weight inference . |
| Outcome: | The proposed inference system achieves 6.25x speed increase over full-precision baselines and 2.32x over low-bit baselines. |
Simplifying Neural Machine Translation with Addition-Subtraction Twin-Gated Recurrent Networks (D18-1)
Copied to clipboard
| Challenge: | Existing gated recurrent networks have a vanishing gradient, allowing for more matrix transformations and less transparent functions. |
| Approach: | They propose an additionsubtraction twin-gated recurrent network (ATR) to simplify neural machine translation. |
| Outcome: | The proposed system is more transparent than LSTM/GRU due to the simplification. |
Speeding Up Neural Machine Translation Decoding by Cube Pruning (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation suffers from slow translation speed due to the large search space . a trade-off has to be made between translation quality and speed, argues a new study . |
| Approach: | They apply cube pruning technique to speed up dynamic programming into neural machine translation to speed it up. |
| Outcome: | The proposed method can translate faster on GPUs and CPUs with better translation quality than naive beam search. |
DecoCal: Decoding with Calibration in Diffusion Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Diffusion Large Language Models (DLLMs) generate text via iterative token denoising . but decoding is challenging, with many tokens appearing predictable early . |
| Approach: | They propose a Decoding framework that performs Calibration of token-level confidence across diffusion steps and leverages the calibrated results to guide decoding decisions. |
| Outcome: | Experiments on multiple DLLMs and benchmarks show that DecoCal improves generation accuracy compared to existing strategies. |
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing transformer models are computationally demanding and prohibitively costly for long sequences due to the quadratic complexity of its selfattention module. |
| Approach: | They propose a transformer-based model that inherits weights from large pretrained models by removing redundancies in hidden sequences using the ready-made Fast Fourier Transform operator. |
| Outcome: | The proposed model outperforms the standard BART model on the long-range modeling benchmark LRA with significant improvements in speed and space. |
MATE: Multi-view Attention for Table Transformer Efficiency (2021.emnlp-main)
Copied to clipboard
| Challenge: | Tables are ubiquitous on the web, and are rich in information. |
| Approach: | They propose a sparse-attention Transformer architecture for modeling documents that contain large tables. |
| Outcome: | The proposed architecture scales linearly with respect to speed and memory, and can handle documents containing more than 8000 tokens with current accelerators. |
Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities in handling long context inputs, but this comes at the cost of increased computational resources and latency. |
| Approach: | They propose an algorithm that uses early LLM layers as filters to select and compress input tokens, reducing the context length for subsequent processing. |
| Outcome: | The proposed method outperforms existing techniques on the Needle in a Haystack task while demonstrating comparable performance on the LongBench challenge. |
Cross-Policy Compliance Detection via Question Answering (2021.emnlp-main)
Copied to clipboard
| Challenge: | Policy compliance detection is the task of ensuring that a scenario conforms to a policy. |
| Approach: | They propose to decompose policy compliance detection into question answering . they propose to use an existing dataset to augment expert annotations . |
| Outcome: | The proposed approach improves accuracy in cross-policy setups, especially when policies are unseen in training. |
BiMax: Bidirectional MaxSim Score for Document-Level Alignment (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Document alignment is necessary for the hierarchical mining of documents across source and target languages. |
| Approach: | They propose a cross-lingual Bidirectional Maxsim score for computing doc-to-doc similarity. |
| Outcome: | The proposed method achieves accuracy comparable to OT with an approximate 100-fold speed increase. |
Evaluating the Consistency of LLM Evaluators (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown potential as general evaluators with the benefits of speed and cost. |
| Approach: | They conduct extensive studies on the two aspects of consistency in LLM evaluations, Self-Consistency (SC) and Inter-scale Consistency on different scoring scales and criterion granularity with open-source and proprietary models. |
| Outcome: | The results show that strong proprietary models are not necessarily consistent evaluators, highlighting the importance of considering consistency in assessing the capability of LLM evalueators. |
Reorder and then Parse, Fast and Accurate Discontinuous Constituency Parsing (2022.emnlp-main)
Copied to clipboard
| Challenge: | Discontinuous constituency parsing is still being developed for its efficiency and accuracy are far behind its continuous counterparts. |
| Approach: | They propose to transform a discontinuous constituent tree into a pseudo-continuous one by reordering words in the sentence. |
| Outcome: | The proposed method can transform a discontinuous constituent tree into a pseudo-continuous one by parsing and performing actions on three classical discontinuous constituency treebanks. |
How Speculative Can Speculative Decoding Be? (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have a largely increased latency due to their ability to autoregressively model . speculative decoding is a technique that trades generation quality for speed . |
| Approach: | They propose to use a draft model to draft tokens autoregressively and then verify them in parallel. |
| Outcome: | The proposed model could draft tokens autoregressively and then verify them in parallel . the proposed model trades quality for speed and could fail in verification stage . |
SecFormer: Fast and Accurate Privacy-Preserving Inference for Transformer Models via SMPC (2024.findings-acl)
Copied to clipboard
| Challenge: | a growing number of cloud-based inference services are relying on SMPC to protect data privacy. |
| Approach: | They propose a framework for Privacy-Preserving Inference for Transformer models that eliminates exponential and maximum operations in PPI without sacrificing model performance. |
| Outcome: | The proposed framework outperforms MPCFormer in terms of performance and efficiency . it is 3.57 and 3.58 times faster than PUMA for BERTBASE and BERTLARGE . |
Reducing Discontinuous to Continuous Parsing with Pointer Network Reordering (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing discontinuous constituent parsers are slow and lack accuracy and speed . however, discontinuous parsing can be solved by any off-the-shelf continuous parser . |
| Approach: | They propose to reduce discontinuous constituent parsing to a continuous problem by reordering tokens. |
| Outcome: | The proposed method is on par with state-of-the-art methods but considerably faster. |
Alignment for Efficient Tool Calling of Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in tool learning have enabled large language models to integrate external tools, enhancing their task performance by expanding their knowledge boundaries. |
| Approach: | They propose a framework that combines probabilistic knowledge boundary estimation with dynamic decision-making to allow LLMs to better assess when to invoke tools based on their confidence. |
| Outcome: | The proposed framework shows significant improvements in tool efficiency by reducing unnecessary tool usage. |
Faster and Better LLMs via Latency-Aware Test-Time Scaling (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research has overlooked the efficiency of TTS from a latency-sensitive perspective. |
| Approach: | They propose two approaches to achieve latency-optimal TTS by branch-wise parallelism and sequence-wise parallelism. |
| Outcome: | The proposed approach achieves latency-optimal TTS for large models . branch-wise parallelism and sequence-wise parallelism are key approaches . |
MATTER: Memory-Augmented Transformer Using Heterogeneous Knowledge Sources (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing retrieval-augmented models typically retrieve information from a single type of knowledge source. |
| Approach: | They propose an efficient memory-augmented transformer to retrieve relevant knowledge from multiple knowledge sources. |
| Outcome: | The proposed model outperforms existing retrieval-augmented models on popular QA benchmarks in terms of accuracy and speed. |
Non-Autoregressive Document-Level Machine Translation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing non-autoregressive translation models struggle with document context and handling discourse phenomena. |
| Approach: | They propose a simple but effective design of sentence alignment between source and target to improve their performance on document-level machine translation. |
| Outcome: | The proposed model achieves high acceleration on documents and sentence alignment significantly enhances their performance. |
Just Put a Human in the Loop? Investigating LLM-Assisted Annotation for Subjective Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown impressive performance in many annotation tasks, including subjective tasks common in content moderation and text analysis in the social sciences. |
| Approach: | They propose to give crowdworkers LLM-generated annotation suggestions to "review" LLMs for subjective tasks can impact model performance and analysis downstream . |
| Outcome: | The proposed approach improves self-reported confidence in annotators and models . it also significantly improves model performance by analyzing human-approved datasets. |
SpecExtend: A Drop-in Enhancement for Speculative Decoding of Long Sequences (2026.findings-acl)
Copied to clipboard
| Challenge: | Speculative decoding performance degrades as input length increases, with significant drops even at moderate lengths. |
| Approach: | They propose a drop-in enhancement that improves speculative decoding on long sequences without additional training. |
| Outcome: | The proposed enhancement accelerates speculative decoding by up to 2.84 on 16K-token long document summarization and up to 3.86 on long-form reasoning while preserving the short-input performance of state-of-the-art frameworks. |