Gatekeeper to save COGS and improve efficiency of Text Prediction (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) require massive amounts of computation and storage, such an approach incurs network and high execution cost. |
| Approach: | They propose a model gatekeeper to stop LLM calls that result in incorrect predictions . they show it can save 46.6% of COGS and improve user experience by not showing incorrect predictions. |
| Outcome: | The proposed model gatekeeper saves 46.6% of COGS and improves user experience . it also improves the suggestion rate of the proposed model by 73% . |
Similar Papers
Collaborative Performance Prediction for Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are one of the most important AI research powered by largescale parameters, high computational resources, and massive training data. |
| Approach: | They propose a framework that leverages historical performance of large language models and other design factors to improve prediction accuracy. |
| Outcome: | The proposed framework surpasses scaling laws in predicting performance of large language models . it also facilitates a detailed analysis of factor importance, an area previously overlooked . |
Cache Saver: A Modular Framework for Efficient, Affordable, and Reproducible LLM Inference (2025.findings-emnlp)
Copied to clipboard
Nearchos Potamitis, Lars Henning Klein, Bardia Mohammadi, Chongyang Xu, Attreyee Mukherjee, Niket Tandon, Laurent Bindschaedler, Akhil Arora
| Challenge: | Inference is the major cost throughout the lifecycle of a large language model (LLM). |
| Approach: | They propose a plug-and-play, asynchronous framework that facilitates high-level inference optimizations. |
| Outcome: | The proposed framework reduces cost and CO2 by 35% across methods, tasks, and LLMs. |
Harnessing Large Language Models as Post-hoc Correctors (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have demonstrated their effectiveness in a wide range of tasks, including machine translation and commonsense reasoning. |
| Approach: | They propose a training-free framework that can work as a post-hoc corrector to propose corrections for ML models. |
| Outcome: | The proposed framework improves the performance of a number of models by up to 39% on text analysis and the challenging molecular predictions. |
Guided by Gut: Efficient Test-Time Scaling with Reinforced Intrinsic Confidence (2026.acl-long)
Copied to clipboard
| Challenge: | Guided by Gut (GG) is an efficient self-guided TTS framework for Large Language Models (LLMs) that performs step-by-step reasoning at a low cost without any reward models or verifiers. |
| Approach: | They propose a self-guided TTS framework that enables LLMs to perform step-by-step reasoning at a low cost without any reward models or verifiers. |
| Outcome: | Empirical evaluations show that GG performs better than TTS with PRMs while reducing GPU memory usage by up to 10. |
MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization (2023.findings-emnlp)
Copied to clipboard
Yuyan Chen, Zhihao Wen, Ge Fan, Zhengyu Chen, Wei Wu, Dayiheng Liu, Zhixu Li, Bang Liu, Yanghua Xiao
| Challenge: | Existing research emphasizes the importance of adapting prompts to specific tasks, rather than specific LLMs. |
| Approach: | They propose a model-adaptive prompt optimizer method that optimizes original prompts for each LLM in downstream tasks. |
| Outcome: | The proposed method can optimize prompts for an LLM in downstream tasks. |
ShieldHead: Decoding-time Safeguard for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in LLM-based moderation methods have demonstrated remarkable promise in identifying safety risks associated with both inputs and outputs in human-AI interactions. |
| Approach: | They propose to learn a classification head on the last-layer hidden states of a dialogue model and use it to detect harmful content. |
| Outcome: | The proposed framework is 300 faster (**1ms**) than previous LLM-based moderation models with 99% less parameters than LlamaGuard. |
Decoupling Task-Solving and Output Formatting in LLM Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Recent studies suggest intertwining task and format instructions with strict formatting requirements can negatively impact LLMs' reasoning capabilities. |
| Approach: | They propose a decoding framework that explicitly decouples format adherence from problem solving. |
| Outcome: | Experiments show that Deco-G consistently gains over prompting and structured generation baselines, with guaranteed format compliance. |
GTA: Gated Toxicity Avoidance for LM Performance Preservation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing Controllable Text Generation methods that generate toxic text can negatively impact the performance of the language model. |
| Approach: | They propose a gated Toxicity Avoidance method that can be applied to any CTG method and evaluate its effectiveness. |
| Outcome: | The proposed method reduces toxicity and preserves performance while preserving the language model's generation performance. |
KoSBI: A Dataset for Mitigating Social Bias Risks Towards Safer Large Language Model Applications (2023.acl-industry)
Copied to clipboard
| Challenge: | Existing research and resources are not readily applicable in South Korea due to the differences in language and culture, both of which significantly affect the biases and targeted demographic groups. |
| Approach: | They propose a social bias dataset of 34k pairs of contexts and sentences in Korean covering 72 demographic groups in 15 categories. |
| Outcome: | The proposed dataset reduces social biases by 16.47%p on average for HyperClova (30B and 82B), and GPT-3. |
Remember what you did so you know what to do next (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have shown large language models (LLMs) to be poor fit for a simulated robot to achieve 30 classes of goals. |
| Approach: | They use the 6B parameter GPT-J language model to create a plan for a simulated robot to achieve 30 classes of goals in ScienceWorld. |
| Outcome: | The proposed model outperforms the state-of-the-art by a factor of 1.4 when training on as many prior steps as will fit, and the results are 2.3x better than the state of the-art. |