Batch Prompting: Efficient Inference with Large Language Model APIs (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Performing inference on large volumes of samples can be computationally and financially costly. |
| Approach: | They propose a prompting approach that enables large language models to run inference in batches instead of one sample at a time. |
| Outcome: | The proposed prompting reduces both token and time costs while retaining downstream performance. |
Similar Papers
More Samples or More Prompts? Exploring Effective Few-Shot In-Context Learning for LLMs with In-Context Sampling (2024.findings-naacl)
Copied to clipboard
Bingsheng Yao, Guiming Chen, Ruishi Zou, Yuxuan Lu, Jiachen Li, Shao Zhang, Yisi Sang, Sijia Liu, James Hendler, Dakuo Wang
| Challenge: | Existing studies on LLM prompting focus on selecting a better set of data samples inside one single prompt input, but why not design and leverage multiple ICL prompts together to further improve the LLM’s performance? |
| Approach: | They propose a low-resource LLM prompting technique to optimize the construction of multiple ICL prompt inputs to produce confident predictions. |
| Outcome: | The proposed technique can produce confident predictions by optimizing the construction of multiple ICL prompt inputs on four NLI datasets and one QA dataset. |
Prompterator: Iterate Efficiently towards More Effective Prompts (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) use a process known as prompting to solve arbitrary language tasks. prompting is a non-trivial task that requires experimentation in order to arrive at a prompt that solves a specific task. |
| Approach: | They propose a tool that helps users iterate over different potential prompts and choose the best performing one based on human feedback. |
| Outcome: | The proposed tool is open source and easily extensible. |
MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization (2023.findings-emnlp)
Copied to clipboard
Yuyan Chen, Zhihao Wen, Ge Fan, Zhengyu Chen, Wei Wu, Dayiheng Liu, Zhixu Li, Bang Liu, Yanghua Xiao
| Challenge: | Existing research emphasizes the importance of adapting prompts to specific tasks, rather than specific LLMs. |
| Approach: | They propose a model-adaptive prompt optimizer method that optimizes original prompts for each LLM in downstream tasks. |
| Outcome: | The proposed method can optimize prompts for an LLM in downstream tasks. |
Instances Need More Care: Rewriting Prompts for Instances with LLMs in the Loop Yields Better Zero-Shot Performance (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have revolutionized zero-shot task performance, mitigating the need for task-specific annotations while enhancing task generalizability. |
| Approach: | They propose an approach that optimizes the zero-shot prompts for individual task instances following an innovative manner of "LLMs in the loop" their results show that PRomPTed outperforms naive zero- shot approaches and a strong baseline which refines the task output instead of the input prompt. |
| Outcome: | The proposed approach outperforms naive approaches and a strong baseline which refines the task output instead of the input prompt. |
LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly lengthy and require longer prompts . this paper presents a coarse-to-fine prompt compression method to reduce cost and increase performance. |
| Approach: | They propose a coarse-to-fine prompt compression method that maintains semantic integrity under high compression ratios and a token-level iterative compression algorithm to better model the interdependence between compressed contents. |
| Outcome: | The proposed method yields state-of-the-art performance and allows for up to 20x compression with little performance loss over four datasets from different scenarios. |
Metacognitive Prompting Improves Understanding in Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in prompting have enhanced reasoning in logic-intensive tasks for LLMs, yet the nuanced understanding abilities of these models remain underexplored. |
| Approach: | They propose a strategy inspired by human introspective reasoning processes to enhance LLMs' understanding abilities. |
| Outcome: | The proposed method outperforms chain-of-thought prompting and its advanced versions on ten natural language understanding (NLU) datasets. |
Soft Prompting for Unlearning in Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing ethical and safety considerations for large language models are important for deployment . however, some ethical concerns have been raised due to the presence of private, sensitive, or harmful information in the training data. |
| Approach: | They propose a framework that learns prompt tokens that are prepended to a query to induce unlearning in LLMs. |
| Outcome: | The proposed method improves the trade-off between utility and forgetting for text classification and question-answering. |
Rethinking the Role of Prompting Strategies in LLM Test-Time Scaling: A Perspective of Probability Theory (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that scaling test-time compute can also effectively improve reasoning. |
| Approach: | They propose a probabilistic method to efficiently predict scaling performance and identify the best prompting strategy under large sampling times. |
| Outcome: | The proposed method significantly improves the scaling performance of majority voting on large language models. |
Multi-expert Prompting Improves Reliability, Safety and Usefulness of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing enhancements of ExpertPrompting improve the large language model generation process. |
| Approach: | They propose a novel enhancement of ExpertPrompting to improve LLM generation by simulating multiple experts, aggregating their responses and selecting the best among individual and aggregated responses. |
| Outcome: | The proposed enhancement outperforms ExpertPrompting and comparable baselines in truthfulness, factuality, informativeness, usefulness and harmfulness. |
Solving Data-centric Tasks using Large Language Models (2024.findings-naacl)
Copied to clipboard
Shraddha Barke, Christian Poelitz, Carina Negreanu, Benjamin Zorn, José Cambronero, Andrew Gordon, Vu Le, Elnaz Nouri, Nadia Polikarpova, Advait Sarkar, Brian Slininger, Neil Toronto, Jack Williams
| Challenge: | Large language models are increasingly useful for data-centric tasks, but how do we decide how much data to include in the prompt? |
| Approach: | They propose a cluster-then-select prompting technique that adds the most representative rows from the input data to the LLM prompt. |
| Outcome: | The proposed technique outperforms a baseline for tasks with syntactic variation in the input table. |