Papers with CPUs
Unsupervised Term Extraction for Highly Technical Domains (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Term extraction is an important task for knowledge discovery platforms because domain specific terms are the linguistic representation of domainspecific concepts. |
| Approach: | They propose a term extraction subsystem that uses an unsupervised annotator to generate training data to fine-tune transformer models. |
| Outcome: | The proposed system can generalize across domains while reducing latency and inference time while preserving the high performance of the existing system. |
Rethinking KenLM: Good and Bad Model Ensembles for Efficient Text Quality Filtering in Large Web Corpora (2025.acl-short)
Copied to clipboard
| Challenge: | Existing methods to efficiently filter large web corpora require GPU resources. |
| Approach: | They propose an ensemble approach that leverages two contrasting KenLMs to filter large web corpora. |
| Outcome: | The proposed method significantly reduces noisy content while preserving high-quality content compared to the traditional KenLM training method. |
EasyJudge: an Easy-to-use Tool for Comprehensive Response Evaluation of LLMs (2025.coling-demos)
Copied to clipboard
| Challenge: | Existing open-source evaluation models lack a user-friendly visualization tool and are not optimized for accelerated model inference. |
| Approach: | They propose to use open-source evaluation models to evaluate language model responses. |
| Outcome: | The proposed model is lightweight, precise, efficient, and user-friendly, with an intuitive visualization interface for ease of deployment and use. |
UltraSparseBERT: 99% Conditionally Sparse Language Modelling (2024.acl-short)
Copied to clipboard
| Challenge: | UltraSparseBERT uses 0.3% of its neurons during inference, compared to similar BERT models. |
| Approach: | They propose a BERT variant that selectively engages just 12 out of 4095 neurons for each layer inference. |
| Outcome: | The proposed model employs 0.3% of its neurons during inference while performing on par with similar models. |
LP Data Pipeline: Lightweight, Purpose-driven Data Pipeline for Large Language Models (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Creating high-quality datasets for large language models often relies on resource-intensive, GPU-accelerated models for quality filtering, making the process time-consuming and costly. |
| Approach: | They propose a framework that operates entirely on CPUs to streamline the processes of dataset extraction, filtering, and curation. |
| Outcome: | The proposed framework reduces preparation time and costs while maintaining high data quality while enhancing the applicability of LLMs in specialized contexts. |
ArcLight: A Lightweight LLM Inference Architecture for Many-Core CPUs (2026.acl-demo)
Copied to clipboard
| Challenge: | Existing frameworks for large language model (LLM) inference on CPUs overlook overhead of cross-NUMA memory access. |
| Approach: | They propose a lightweight LLM inference architecture designed from the ground up for many-core CPUs. |
| Outcome: | Experimental results show that ArcLight surpasses the performance ceiling of mainstream frameworks, achieving up to 46% higher inference throughput. |
CUPID: Curriculum Learning Based Real-Time Prediction using Distillation (2023.acl-industry)
Copied to clipboard
| Challenge: | Relevance in E-commerce Product Search is crucial for providing customers with accurate results that match their query intent. |
| Approach: | They propose a curriculum learning based real-time relevance prediction using distillation . they propose e-commerce search systems that use transformers to predict relevance . |
| Outcome: | The proposed model improves on english and Arabic in a bi-lingual relevance prediction task while maintaining low evaluation latency on CPUs. |
Sparse Distillation: Speeding Up Text Classification by Using Bigger Student Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to reduce inference cost by distilling transformer models into lightweight student models are limited for high-volume use cases. |
| Approach: | They propose to distill state-of-the-art transformer models into lightweight student models to reduce computation cost at inference time. |
| Outcome: | The proposed pipeline achieves up to 600x speed-up on GPUs and CPUs on six single-sentence text classification tasks and in domain generalization settings. |
Scalable Fine-tuning from Multiple Data Sources: A First-Order Approximation Approach (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a new algorithm to estimate fine-tuning performance for a target task is proposed . conventional subset selection methods require repeated training on subsets of auxiliary tasks . |
| Approach: | They propose an algorithm to fine-tune a language model for a target task by optimally using auxiliary tasks' information. |
| Outcome: | The proposed method can estimate fine-tuning performance on CPUs in seconds. |
Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index (P19-1)
Copied to clipboard
| Challenge: | Existing open-domain question answering models require multiple documents on-demand for every input query. |
| Approach: | They propose query-agnostic indexable representations of document phrases that can drastically speed up open-domain question answering. |
| Outcome: | The proposed model can be trained and deployed even in a single 4-GPU server. |
Speeding Up Neural Machine Translation Decoding by Cube Pruning (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation suffers from slow translation speed due to the large search space . a trade-off has to be made between translation quality and speed, argues a new study . |
| Approach: | They apply cube pruning technique to speed up dynamic programming into neural machine translation to speed it up. |
| Outcome: | The proposed method can translate faster on GPUs and CPUs with better translation quality than naive beam search. |
Learning Dense Representations of Phrases at Scale (2021.acl-long)
Copied to clipboard
| Challenge: | Existing phrase retrieval models rely on sparse representations and still underperform retriever-reader approaches. |
| Approach: | They propose a method to learn phrase representations from reading comprehension tasks using negative sampling methods. |
| Outcome: | The proposed model improves over previous models by 15%-25% absolute accuracy and matches the performance of state-of-the-art retrieval models. |
Dovetail: A CPU/GPU Heterogeneous Speculative Decoding for LLM inference (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are demanding more memory and computational resources . however, these devices typically feature weaker GPUs and stronger CPUs . |
| Approach: | They propose a lossless inference acceleration method that leverages the characteristics of heterogeneous devices and the advantages of speculative decoding. |
| Outcome: | The proposed method achieves speedups ranging from 1.79 to 10.1 across different devices . it uses a draft model on the GPU to perform preliminary predictions, while a target model on CPU validates these outputs . |
Gold-Medal-Level Olympiad Geometry Solving with Efficient Heuristic Auxiliary Constructions (2026.acl-long)
Copied to clipboard
Boyan Duan, Xiao Liang, Shuai Lu, Yaoxiang Wang, Yelong Shen, Kai-Wei Chang, Ying Nian Wu, Mao Yang, Weizhu Chen, Yeyun Gong
| Challenge: | Existing methods for geometry theorem proving in Euclidean geometry are challenging and require a neural network to perform. |
| Approach: | They propose a method for adding auxiliary points in geometry that runs on CPUs without relying on neural network-based inference. |
| Outcome: | The proposed method achieves silver-medal-level human performance on IMO-30 benchmark. |