Query-OPT: Optimizing Inference of Large Language Models via Multi-Query Instructions in Meeting Summarization (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing LLMs require a new call to the inference endpoint/API for each new query . repeated calls to the endpoints/AP Is expensive and impractical for many real-world use cases. |
| Approach: | They compare the performance of various LLMs for query-based meeting summarization . they find that combining queries for the same context in a single prompt can be used to minimize repeated calls. |
| Outcome: | The proposed approach reduces the number of calls to the inference endpoints/APIs in meeting summarization tasks. |
Similar Papers
Tell me what I need to know: Exploring LLM-based (Personalized) Abstractive Multi-Source Meeting Summarization (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for meeting summarization rely on transcripts and generate generic summaries, failing to contextualize long discussions and to tailor information to individual preferences and productivity requirements. |
| Approach: | They propose a multi-source approach that considers supplementary materials and generates a summary from this enriched transcript. |
| Outcome: | The proposed model increases summary relevance by 9% and produces more content-rich outputs. |
Batch Prompting: Efficient Inference with Large Language Model APIs (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Performing inference on large volumes of samples can be computationally and financially costly. |
| Approach: | They propose a prompting approach that enables large language models to run inference in batches instead of one sample at a time. |
| Outcome: | The proposed prompting reduces both token and time costs while retaining downstream performance. |
When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have shifted focus toward scaling inference-time compute. |
| Approach: | They propose to scale inference-time compute in a multilingual, multi-task setting . they propose to use m-ArenaHard-v2.0 prompts to sample multiple outputs in parallel . |
| Outcome: | The proposed solutions achieve an average +6.8 jump in win-rates for 8B models on m-ArenaHard-v2.0 prompts in non-English languages against proprietary models like Gemini. |
QuickLLaMA: Query-aware Inference Acceleration for Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) struggle with capturing long-distance dependencies within sequences to deeply understand semantics. |
| Approach: | They propose a system that captures relevant information within a fixed window size and provides precise answers to queries. |
| Outcome: | The proposed system can read Harry Potter within 30s and accurately answer the questions. |
Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval (2024.naacl-long)
Copied to clipboard
| Challenge: | et al., 2020: performance of dense retrieval models in multilingual retrieval is limited due to uneven and scarce training data available across multiple languages. |
| Approach: | They propose a synthetic retrieval training dataset containing 33 languages for fine-tuning multilingual retrievers without human supervision. |
| Outcome: | The proposed model outperforms human-supervised retrieval models on three retrieval benchmarks. |
SkyLLM: Cross-LLM-APIs Federation for Cost-effective Query Processing (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated exceptional capabilities across a wide range of tasks, from text generation to complex problem-solving. |
| Approach: | They propose a system which federates multiple LLM APIs and dynamically assigns a non-empty subset of these APIs to each query prior to inference. |
| Outcome: | The proposed system can match the most accurate LLM with the lowest cost while cutting costs by 67.8%. |
SelectLLM: Query-Aware Efficient Selection Algorithm for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing large language models struggle with complex tasks such as factually-grounded reasoning and planning due to inherent training biases, model size constraints, and the quality or diversity of pre-training datasets. |
| Approach: | They propose a novel algorithm to select the most suitable LLMs from a large pool and use it to efficiently generalize and perform tasks. |
| Outcome: | The proposed model outperforms existing ensemble-based baselines and achieves competitive performance with similarly sized top-performing LLMs while maintaining efficiency. |
QUILL: Query Intent with Large Language Models using Retrieval Augmentation and Multi-stage Distillation (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive results on a variety of text understanding tasks. |
| Approach: | They propose a two-stage distillation approach that allows retrieval augmentation to be carried over without the increased compute associated with it. |
| Outcome: | The proposed approach can carry over the gains of retrieval augmentation without suffering the increased compute typically associated with it. |
Solving Data-centric Tasks using Large Language Models (2024.findings-naacl)
Copied to clipboard
Shraddha Barke, Christian Poelitz, Carina Negreanu, Benjamin Zorn, José Cambronero, Andrew Gordon, Vu Le, Elnaz Nouri, Nadia Polikarpova, Advait Sarkar, Brian Slininger, Neil Toronto, Jack Williams
| Challenge: | Large language models are increasingly useful for data-centric tasks, but how do we decide how much data to include in the prompt? |
| Approach: | They propose a cluster-then-select prompting technique that adds the most representative rows from the input data to the LLM prompt. |
| Outcome: | The proposed technique outperforms a baseline for tasks with syntactic variation in the input table. |
Exploring the Best Practices of Query Expansion with Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are foundational in language technologies, particularly in information retrieval (IR). |
| Approach: | They propose a framework that leverages large language models for query expansion . they use LLMs to generate multiple pseudo-references and integrate them with original queries . |
| Outcome: | The proposed framework enhances sparse and dense retrieval methods without pre-indexing. |