COST

AB/BA analysis: A framework for estimating keyword spotting recall improvement while maintaining audio privacy (2022.naacl-industry)

Challenge:	Hardware-in-the-Loop (HIL) testing is essential for automotive validation but suffers from fragmented and underutilized test artifacts.
Approach:	They propose to integrate semantic retrieval with domain-adapted large language models to support test engineers in real-world HIL workflows.
Outcome:	The proposed system improves perceived helpfulness, truthfulness, and satisfaction over general-purpose LLMs.

Copied to clipboard

Raphael Petegrosso, VasistaKrishna Baderdinnni, Thibaud Senechal, Benjamin Bullough

OpenOmni: A Collaborative Open Source Tool for Building Future-Ready Multimodal Conversational Agents (2024.emnlp-demo)

Challenge:	Keyword spotting systems that detect keywords in speech are difficult to evaluate under privacy constraints.
Approach:	They propose to use offline decoding to evaluate a candidate KWS model against a baseline model without requiring negative examples.
Outcome:	The proposed method improves the time, privacy, and cost of the evaluation and compares with real data.

Copied to clipboard

Qiang Sun, Yuanyi Luo, Sirui Li, Wenxiao Zhang, Wei Liu

NextGen AML: Distributed Deep Learning based Language Technologies to Augment Anti Money Laundering Investigation (P18-4)

Challenge:	OpenOmni is an open-source, end-to-end pipeline benchmarking tool for multimodal conversational agents.
Approach:	They developed an open-source, end-to-end pipeline benchmarking tool to help solve these issues.
Outcome:	OpenOmni integrates speech-to-text, emotion detection, and large language models with the ability to integrate customized models.

Copied to clipboard

Jingguang Han, Utsab Barman, Jeremiah Hayes, Jinhua Du, Edward Burgin, Dadong Wan

Challenge:	Money laundering (AML) is the process of transferring criminal and illegal proceeds into ostensibly legitimate assets.
Approach:	They propose a framework that uses deep learning to augment AML monitoring and investigation . money laundering is the process of transferring criminal and illegal proceeds into ostensibly legitimate assets .
Outcome:	The proposed framework reduces time and cost by 30% compared to existing methods . money laundering is the world's third largest "industry"

ConvLab-3: A Flexible Dialogue System Toolkit Based on a Unified Data Format (2023.emnlp-demo)

Copied to clipboard

Qi Zhu, Christian Geishauser, Hsien-chin Lin, Carel van Niekerk, Baolin Peng, Zheng Zhang, Shutong Feng, Michael Heck, Nurul Lubis, Dazhen Wan, Xiaochen Zhu, Jianfeng Gao, Milica Gasic, Minlie Huang

Robustness in NLP

Challenge:	Existing tools for building TOD systems often lack a user-friendly interface . a toolkit with advanced, easily integrable modules is needed to bridge this gap .
Approach:	They propose a multifaceted dialogue system toolkit that integrates diverse datasets and models with a streamlined training process and in-depth evaluation tools.
Outcome:	The proposed toolkit combines RL and transfer learning to support the rapid development and evaluation of robust dialogue policies.

Multi-Tenant Optimization For Few-Shot Task-Oriented FAQ Retrieval (2022.emnlp-industry)

Copied to clipboard

Asha Vishwanathan, Rajeev Warrier, Gautham Vadakkekara Suresh, Chandra Shekhar Kandpal

Low-Resource NLP

Challenge:	Business-specific Frequently Asked Questions (FAQs) are important part of many taskoriented dialog systems today.
Approach:	They propose a method to scale multi-tenant FAQ applications in real-world context . they use query-Question similarity and few-shot intent detection techniques .
Outcome:	The proposed approach scales multi-tenant FAQ applications in real-world context . it uses fine-tuning of the last layer in sentence Bi-Encoders along with tenant-specific weight switching.

Generalising LLM Routing using Past Performance Retrieval: A Few-Shot Router is Sufficient (2026.eacl-srw)

Copied to clipboard

Clovis Varangot-Reille, Christophe Bouvard, Antoine Gourru

EduPulse: A Practical LLM-Enhanced Opinion Mining System for Vietnamese Student Feedback in Educational Platforms (2026.eacl-industry)

Challenge:	Large Language Models (LLMs) use a single LLM to perform tasks.
Approach:	They propose a meta-evaluation framework that predicts per-model performance for new queries by retrieving similar past queries and reweighting model scores with lightweight attention.
Outcome:	The proposed framework matches the quality–cost trade-offs of generalisable routers across five routing benchmarks.

Copied to clipboard

Nguyen Xuan Phuc, Phi Nguyen Xuan, Vinh-Tiep Nguyen, Thìn Dang Van, Ngan Luu-Thuy Nguyen

Sentiment Analysis

Challenge:	EduPulse is a system designed specifically to analyze student feedback in Vietnamese.
Approach:	They propose a system that analyzes student feedback in Vietnamese to improve opinion mining.
Outcome:	The proposed system performs four opinion analysis tasks in Vietnamese . it is scalable and maintainable, and it is cost-effective, the authors show .

The Price of Thought: A Multilingual Analysis of Reasoning, Performance, and Cost of Negotiation in Large Language Models (2026.findings-eacl)

Copied to clipboard

Sherzod Hakimov, Roland Bernard, Tim Leiber, Karl Osswald, Kristina Richert, Ruilin Yang, Raffaella Bernardi, David Schlangen

Reasoning

Challenge:	Negotiation is a fundamental challenge for AI agents as it requires an ability to reason strategically, model opponents, and balance cooperation with competition.
Approach:	They propose to use a self-play setup to compare commercial and open-weight large language models to their vanilla counterparts in three different languages to examine trade-offs between performance and cost.
Outcome:	The proposed model improves GPT-5's performance by 31.4 % while increasing its cost by nearly 400 %.

Selective Perception: Learning Concise State Descriptions for Language Model Actors (2024.naacl-short)

Copied to clipboard

Kolby Nottingham, Yasaman Razeghi, Kyungmin Kim, Jb Lanier, Pierre Baldi, Roy Fox, Sameer Singh

Building Real-World Meeting Summarization Systems using Large Language Models: A Practical Perspective (2023.emnlp-industry)

Challenge:	Recent large language models support longer contexts, but requiring them to process redundant or irrelevant data increases inference time and cost.
Approach:	They propose a method that leverages a small finetuned LM to sample the minimal set of input features that maximizes the performance of a downstream LM.
Outcome:	The proposed method reduces the length of LM actor input by 87% and 99% while improving task success rates by 158% and 54% on NetHack and robot planning.

Copied to clipboard

Md Tahmid Rahman Laskar, Xue-Yong Fu, Cheng Chen, Shashi Bhushan TN

Luna: A Lightweight Evaluation Model to Catch Language Model Hallucinations with High Accuracy and Low Cost (2025.coling-industry)

Challenge:	a study examines how to build meeting summarization systems using large language models . closed-source models are generally better in terms of performance, but open-source ones are more advantageous for industrial use .
Approach:	They compare closed-source and open-source meeting summarization models for real-world use . they find that closed-sourced models are generally better in terms of performance . however, smaller open-sourced LLMs could still achieve comparable performance if they are open .
Outcome:	The proposed model is more efficient for industrial use than closed-source models due to privacy concerns and high cost.

Copied to clipboard

Masha Belyi, Robert Friel, Shuai Shao, Atindriyo Sanyal

SMILE: Single-turn to Multi-turn Inclusive Language Expansion via ChatGPT for Mental Health Support (2024.findings-emnlp)

Challenge:	Retrieval-augmented generation (RAG) systems are crucial for enhancing the capabilities of large language models (LLMs) in industry applications.
Approach:	They propose a DeBERTA-large encoder for hallucination detection in RAG settings that is fine-tuned for halluination detection.
Outcome:	The proposed model outperforms GPT-3.5 and commercial evaluation frameworks on the hallucination detection task, with 97% and 91% reduction in cost and latency, respectively.

Copied to clipboard

Huachuan Qiu, Hongliang He, Shuai Zhang, Anqi Li, Zhenzhong Lan

Dialogue Systems & Conversational Agents

Challenge:	Developing specialized dialogue systems for mental health support requires multi-turn conversation data . data privacy protection, time and cost involved in crowdsourcing are challenges . a new method for rewriting public single-turn dialogues into multi-turned ones is needed .
Approach:	They propose a single-turn to multi-turn inclusive language expansion technique that prompts ChatGPT to rewrite public single-turned dialogues into multi-turned ones.
Outcome:	The proposed method generates a large-scale, lifelike, and diverse dialogue dataset . it also develops SMILECHAT, a mental health chatbot .

HULK: An Energy Efficiency Benchmark Platform for Responsible Natural Language Processing (2021.eacl-demos)

Copied to clipboard

Xiyou Zhou, Zhiyu Chen, Xiaoyong Jin, William Yang Wang

TokCollate: A Comprehensive Tool for Tokenizer Evaluation and Visualization across Languages (2026.acl-demo)

Challenge:	Pretrained models have been taking the lead of many natural language processing benchmarks such as GLUE, but energy efficiency in the process of model training and inference becomes a critical bottleneck.
Approach:	They propose a multi-task energy efficiency benchmarking platform for responsible natural language processing that compares pretrained models’ energy efficiency from the perspectives of time and cost.
Outcome:	The proposed model improves on the fine-tuning efficiency of pretrained models from the perspectives of time and cost.

Copied to clipboard

Dušan Variš, Abishek Stephen, Jindřich Libovický

Syntactic Text Processing

Challenge:	Tokenization quality varies significantly across languages, leading to disparities in LLM performance and cost for speakers of less-resourced languages.
Approach:	They propose a Python-based evaluation framework and a JavaScript visualization interface that evaluates tokenizers in a variety of languages.
Outcome:	TokCollate is an evaluation framework for tokenizers with a JavaScript visualization interface.

Beyond Abstracts: A New Dataset, Prompt Design Strategy and Method for Biomedical Synthesis Generation (2024.acl-srw)

Copied to clipboard

James O’Doherty, Cian Nolan, Yufang Hou, Anya Belz

A Simple yet Efficient Prompt Compression Method for Text Classification Data Annotation Using LLM (2025.coling-industry)

Challenge:	Existing methods to automate systematic reviews of papers are slow and incomplete . authors propose a new method to automating the systematic review process .
Approach:	They propose a method for automatic synthesis generation using a dataset and prompting-based method.
Outcome:	The proposed method improves the existing model and prompts the system to generate high-quality syntheses.

Copied to clipboard

Yiran Xie, Debin Xiao, Ping Wang, Shuming Liu

SAGE: A Search-AuGmented Evaluation of Large Language Models on Free-Form QA (2026.acl-long)

Challenge:	Existing methods to improve the accuracy of large language models (LLMs) are often impractical due to high costs and time consumption.
Approach:	They propose a method that uses keyword extraction to reduce prompt tokens in text annotation tasks.
Outcome:	The proposed method reduces prompt tokens while maintaining high accuracy.

Copied to clipboard

Sher Badshah, Ali Emami, Hassan Sajjad

SchemaRAG: Dynamic Large Schema Reduction for LLM-driven Structured Information Extraction (2026.acl-industry)

Challenge:	Large Language Models (LLMs) are prone to hallucination and rely on static, pre-annotated references for evaluation.
Approach:	They propose a framework to assess large language models without fixed ground-truth answers by iteratively generating web queries and synthesizing external evidence.
Outcome:	The proposed framework achieves substantial to perfect agreement with human evaluations on multiple free-form QA benchmarks.

Copied to clipboard

Sin Yu Bonnie Ho, Arlie Coles, Erik Larsson, Eric Marshall, Nathan Bodenstab, Paul Vozila

Information Extraction & Text Mining

Challenge:	Structured information extraction (IE) pairs values from unstructured text with schema-defined keys.
Approach:	They propose a retrieval-augmented generation framework that prunes the output schema space for schema-conditioned information extraction tasks by leveraging schema metadata and few-shot examples.
Outcome:	The proposed framework can achieve up to 8.8% increase in micro-F1, 47% reduction in latency, and 48% reduction in token costs on real-world healthcare and e-commerce datasets.

Invoke Interfaces Only When Needed: Adaptive Invocation for Large Language Models in Question Answering (2025.findings-emnlp)

Copied to clipboard

Jihao Zhao, Chunlai Zhou, Daixuan Li, Shuaishuai Zu, Biao Qin

Reasoning

Challenge:	a new metric is developed to pinpoint the moment of invocation when hallucinations arise in small LMs.
Approach:	They propose a metric that measures hallucinations during the generation process of small LMs.
Outcome:	The proposed metric outperforms baselines in hallucination detection across multiple QA datasets.

CascadeDebate: Multi-Agent Deliberation for Cost-Aware LLM Cascades (2026.acl-industry)

Copied to clipboard

Raeyoung Chang, Dongwook Kwon, Jisoo Lee, Nikhil Verma

Challenge:	Large language models (LLMs) have demonstrated remarkable proficiency across diverse benchmarks, spanning scientific question answering to medical diagnosis tasks.
Approach:	They propose to insert multi-agent deliberation directly at each tier’s escalation boundary to enable consensus-driven resolution of ambiguities internally without invoking higher-cost upgrades.
Outcome:	The proposed architecture outperforms strong single-model cascades and standalone multi-agent systems across five benchmarks spanning science, medicine, and general knowledge by up to 26.75%.

MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL Translation (2025.naacl-long)

Copied to clipboard

Satya Krishna Gorti, Ilan Gofman, Zhaoyan Liu, Jiapeng Wu, Noël Vouitsis, Guangwei Yu, Jesse C. Cresswell, Rasa Hosseinzadeh

Text Generation

Challenge:	Recent advances in text-to-SQL generation rely on large closed-source models that present challenges in accessibility, privacy, and latency.
Approach:	They propose to use open-source text-to-SQL models to critique SQL queries . their method evaluates multiple outputs simultaneously and is competitive with larger models .
Outcome:	The proposed method achieves state-of-the-art performance compared to open-source models while remaining competitive with larger models at a much lower cost.

Rethinking Scale: Deployment Trade-offs of Small Language Models under Agent Paradigms (2026.acl-industry)

Copied to clipboard

Xinlin Wang

Talk to Your Slides: High-Efficiency Slide Editing via Language-Driven Structured Data Manipulation (2026.findings-acl)

Challenge:	Existing research focuses on enhancing large language models through scaling laws or fine-tuning strategies, but ignores the potential of using agent paradigms to compensate for the inherent weaknesses of small models.
Approach:	They propose to use structured agent frameworks to improve effectiveness over direct prompting . they also propose to employ routing-based multi-agent systems with collaborative capabilities .
Outcome:	The proposed model significantly outperforms direct prompting with single-agent systems . the proposed model is more reliable and cost-effective than other models .

Copied to clipboard

Kyudan Jung, Hojun Cho, Jooyeol Yun, Soyoung Yang, Jaehyeok Jang, Jaegul Choo

Confidence-Calibrated Small-Large Language Model Collaboration for Cost-Efficient Reasoning (2026.eacl-long)

Challenge:	Talk-to-Your-Slides is a high-efficiency slide editing agent that uses language-driven structured data manipulation instead of the image modality.
Approach:	They propose a language-driven slide editing agent that uses language-based structured data manipulation instead of image modality.
Outcome:	The proposed system achieves faster processing and better instruction fidelity than GUI-based agents.

Copied to clipboard

Chuang Zhang, Zizhen Zhu, Yihao Wei, Bing Tian, Junyi Liu, Henan Wang, Wang Xavier, Yaxiao Liu

Performance-Guided LLM Knowledge Distillation for Efficient Text Classification at Scale (2024.emnlp-main)

Challenge:	Large language models (LLMs) have superior reasoning capabilities compared to small language models, but incur substantially higher inference costs.
Approach:	They propose a system that cascades an LLM with an SLM to achieve a balance between accuracy and cost in complex reasoning tasks.
Outcome:	The proposed system improves the SLM’s reasoning ability and confidence calibration across diverse datasets and model backbones.

Copied to clipboard

Flavio Palo, Prateek Singhi, Bilal Fadlallah

Information Retrieval

Challenge:	Large Language Models (LLMs) face high computational demands at inference time due to high computational costs.
Approach:	They propose a cost-effective and high-throughput solution for large language models . PGKD distills the knowledge of LLMs into smaller, task-specific models based on teacher-student knowledge distillation .
Outcome:	PGKD outperforms BERT-based models and other knowledge distillation methods on multi-class classification datasets.

Improving Reward Models with Synthetic Critiques (2025.findings-naacl)

Copied to clipboard

Zihuiwen Ye, Fraser David Greenlee, Max Bartolo, Phil Blunsom, Jon Ander Campos, Matthias Gallé

Robustness in NLP

Challenge:	a recent study shows that reward models overfit on superficial features, hindering generalization performance . prevailing approach to training preference-based reward models presents several challenges .
Approach:	They propose a method that uses synthetic natural language critiques to provide additional feedback to large language models.
Outcome:	The proposed approach improves performance and data efficiency of RMs initialized from different pretrained models, reducing the reliance on costly human annotations.

Select-Then-Decompose: From Empirical Analysis to Adaptive Selection Strategy for Task Decomposition in Large Language Models (2025.emnlp-main)

Copied to clipboard

Shuodi Liu, Yingzhuo Liu, Zi Wang, Yusheng Wang, Huijia Wu, Liuyu Xiang, Zhaofeng He

Neural network embeddings recover value dimensions from psychometric survey items on par with human data (2026.findings-eacl)

Challenge:	Existing task decomposition methods focus on memory, tool usage, and feedback mechanisms, but they often overlook the trade-off between performance and cost.
Approach:	They propose a strategy that selects the most suitable decomposition approach based on task characteristics and enhances the reliability of the results through a verification module.
Outcome:	The proposed strategy is based on categories of approaches, characteristics of tasks, and configuration of decomposition and execution models.

Copied to clipboard

Max Pellert, Clemens M Lechner, Indira Sen, Markus Strohmaier

Representation Learning

Challenge:	Embedings from large language models can recover structure of human values . quantitative analysis reveals that SQuID addresses the challenge of obtaining negative correlations between dimensions without domain-specific fine-tuning or training data reannotation.
Approach:	They propose to use questionnaire item embeddings to recover human values from PVQ-RR . their results have implications for psychometrics and social science research .
Outcome:	The proposed method explains 55% variance in dimension-dimension similarities compared to human data.

InferenceDynamics: Adaptive LLM Routing through Structured Capability and Knowledge Profiling (2026.acl-long)

Copied to clipboard

Haochen Shi, Tianshi Zheng, Weiqi Wang, Baixuan Xu, Chunyang Li, Chunkit Chan, Tao Fan, Yangqiu Song

Structuring Radiology Reports: Challenging LLMs with Lightweight Models (2025.emnlp-main)

Challenge:	Large Language Model (LLM) routing is a pivotal technique for navigating a diverse landscape of LLMs.
Approach:	They propose a flexible and scalable multi-dimensional routing framework that models the capability and knowledge of models.
Outcome:	The proposed framework can be used to generalize and identify top-performing models for group-level routing using modern benchmarks including MMLU-Pro, GPQA, BigGenBench, and LiveBench.

Copied to clipboard

Johannes Moll, Louisa Fay, Asfandyar Azhar, Sophie Ostmeier, Sergios Gatidis, Tim C. Lueth, Curtis Langlotz, Jean-Benoit Delbrouck

Effective Pretraining Objectives for Transformer-based Autoencoders (2022.findings-emnlp)

Challenge:	Radiology reports lack a standardized format, limiting both interpretability and machine learning applications.
Approach:	They propose to use lightweight encoder-decoder models for structuring radiology reports . they compare models with eight open-source LLMs with prompting and in-context learning .
Outcome:	The proposed models outperform eight open-source LLMs on a human-annotated test set.

Copied to clipboard

Luca Di Liello, Matteo Gabburo, Alessandro Moschitti

Multi-Layered Evaluation Using a Fusion of Metrics and LLMs as Judges in Open-Domain Question Answering (2025.coling-main)

Challenge:	ELECTRA is more accurate than BERT, but it is not clear if this is due to its innovative architecture or to the long and extensive training, which highly increases the computation cost for obtaining the final language model.
Approach:	They propose to replace BERT’s Masked Language Modeling objective (MLM) with Token Detection (TD) by using a statistical approach to generate light tokens.
Outcome:	The proposed method can replace ELECTRA's computationally heavy generators without a significant drop in performance.

Copied to clipboard

Rashin Rahnamoun, Mehrnoush Shamsfard

Natural Language Interfaces

Challenge:	Existing methods for comparing machine-generated answers with reference are not perfect in terms of accuracy or cost.
Approach:	They propose to summarize long answers and use shortened versions to improve evaluation . they propose a multi-layered evaluation methodology that integrates different metrics tailored to various scenarios .
Outcome:	The proposed method outperforms existing evaluation methods but is more cost-effective than existing methods.

Semi-Supervised Reward Modeling via Iterative Self-Training (2024.findings-emnlp)

Copied to clipboard

Yifei He, Haoxiang Wang, Ziyan Jiang, Alexandros Papangelis, Han Zhao

SATER: A Self-Aware and Token-Efficient Approach to Routing and Cascading (2025.emnlp-main)

Challenge:	Reward models capture values and preferences of humans and are used in Reinforcement Learning with Human Feedback (RLHF) Traditionally, training large language models relies on extensive human-annotated preference data, which poses significant challenges in terms of scalability and cost.
Approach:	They propose a method that enhances RM training using unlabeled data.
Outcome:	The proposed approach improves reward models without incurring additional labeling costs on unlabeled datasets.

Copied to clipboard

Yuanzhe Shen, Yide Liu, Zisu Huang, Ruicheng Yin, Xiaoqing Zheng, Xuanjing Huang

MixLLM: Dynamic Routing in Mixed Large Language Models (2025.naacl-long)

Challenge:	Large language models (LLMs) demonstrate remarkable performance across diverse tasks, yet their effectiveness often depends on costly commercial APIs or cloud services.
Approach:	They propose a dual-mode compatible approach that fine-tunes models through shortest-response preference optimization and a confidence-aware rejection mechanism.
Outcome:	The proposed approach reduces redundant outputs and response times while reducing computational costs by over 50% and cascade latency by over 80%.

Copied to clipboard

Xinyuan Wang, Yanchi Liu, Wei Cheng, Xujiang Zhao, Zhengzhang Chen, Wenchao Yu, Yanjie Fu, Haifeng Chen

RouteMoA: Dynamic Routing without Pre-Inference Boosts Efficient Mixture-of-Agents (2026.acl-long)

Challenge:	Large Language Models (LLMs) exhibit potential artificial generic intelligence, however, their usage is costly with high response latency.
Approach:	They develop a dynamic contextual-bandit-based routing system for query-LLM assignment that leverages query tags to enhance query embeddings.
Outcome:	The proposed model maximizes response quality and minimizes cost and latency.

Copied to clipboard

Jize Wang, Han Wu, Zhiyuan You, Yiming Song, Yijun Wang, Zifei Shan, Yining Li, Songyang Zhang, Xinyi Le, Cailian Chen, Xinping Guan, Dacheng Tao

Select to Know: An Internal-External Knowledge Self-Selection Framework for Domain-Specific Question Answering (2025.findings-emnlp)

Challenge:	Existing methods for mixing-of-agents (MoA) lack model selection criteria and struggle with large model pools.
Approach:	They propose a mixture-of-agents framework with dynamic routing that uses a lightweight scorer to perform initial screening and refines the model scores through self- and cross-assessment.
Outcome:	The proposed framework outperforms existing methods for large model pools and tasks . it reduces cost by 89.8% and latency by 63.6% in the large-scale model pool.

Copied to clipboard

Bolei He, Xinran He, Run Shao, Shanfu Shu, Xianwei Xue, MingQuan Cheng, Haifeng Li, Zhen-Hua Ling

Knowledge Representation

Challenge:	Large Language Models (LLMs) perform well in general QA but often struggle in domain-specific scenarios.
Approach:	They propose a framework that internalizes domain knowledge through internal-external knowledge self-selection and selective supervised fine-tuning.
Outcome:	The proposed framework outperforms existing methods and matches domain-pretrained LLMs with significantly lower cost.

User-side Model Consistency Monitoring for Open Source Large Language Models Inference Services (2025.acl-long)

Copied to clipboard

Qijun Miao, Zhixuan Fang

Modeling Legal Reasoning: LM Annotation at the Edge of Human Agreement (2023.emnlp-main)

Challenge:	Open-source large language models have attracted substantial user base by offering quality comparable to closed-source models at a significantly lower cost.
Approach:	They propose a paradigm for model consistency monitoring on the user side that constructs metrics based on logits produced by LLMs to differentiate sequences generated by degraded models.
Outcome:	The proposed method is implementable on consumer-grade devices and demonstrates that it is cost-effective and reliable.

Copied to clipboard

Rosamond Thalken, Edward Stiglitz, David Mimno, Matthew Wilkens

IntroLM: Introspective Language Models via Prefilling-Time Self-Evaluation (2026.findings-acl)

Challenge:	Existing research examines simple classification tasks, but ability of LMs to classify on complex tasks is less well understood.
Approach:	They analyze a Supreme Court opinion annotated by a team of domain experts . they find generative models perform poorly when given instructions equal to human annotators .
Outcome:	The proposed model performs poorly when given instructions equal to instructions given to human annotations . strongest results derive from fine-tuning models on the annotated dataset .

Copied to clipboard

Hossein Hosseini Kasnavieh, Gholamreza Haffari, Christopher Leckie, Adel N. Toosi

When Attention Meets Fast Recurrence: Training Language Models with Reduced Compute (2021.emnlp-main)

Challenge:	Existing methods to predict output quality of large language models rely on external classifiers with limited context windows and constrained representational capacity.
Approach:	They propose a method that enables causal language models to predict their own output quality during the prefilling phase without affecting generation using [CPX] tokens.
Outcome:	The proposed method outperforms existing classifiers on Qwen3-8B and DeBERTa-v3-Large models by 14% on question-answering benchmarks.

Copied to clipboard

Tao Lei

SkewRoute: Training-Free LLM Routing for Knowledge Graph Retrieval-Augmented Generation via Score Skewness of Retrieved Context (2025.findings-emnlp)

Challenge:	Large language models are difficult to train because of the growing computation time and cost.
Approach:	They propose a highly-efficient architecture that combines fast recurrence and attention for sequence modeling.
Outcome:	The proposed model achieves state-of-the-art on a Wiki-103 and Billion Word datasets using 1.6 days of training on an 8-GPU machine.

Copied to clipboard

Hairu Wang, Yuan Feng, Yukun Cao, Xike Xie, S Kevin Zhou

Democratizing LLMs: An Exploration of Cost-Performance Trade-offs in Self-Refined Open-Source Models (2023.findings-emnlp)

Challenge:	Large language models incur high inference costs during deployment, causing hallucination . no dedicated routing methods exist for RAG, and existing training-based routers face challenges scaling to this domain .
Approach:	They propose a plug-and-play routing framework that optimizes performance and cost . the framework delivers over 3x higher routing effectiveness while reducing runtime to less than 0.001x .
Outcome:	The proposed framework delivers over 3x higher routing effectiveness while reducing runtime to less than 0.001x compared to existing methods.

Copied to clipboard

Sumuk Shashidhar, Abhinav Chinta, Vaibhav Sahai, Zhenhailong Wang, Heng Ji

Fast-and-Frugal Text-Graph Transformers are Effective Link Predictors (2025.findings-acl)

Challenge:	Large Language Models (LLMs) have been experiencing seismic growth in size and capabilities, radically transforming the field of NLP.
Approach:	They propose a generalized variant of iterative self-critique and self-refinement devoid of external influence and a ranking metric to find the optimal model for a given task considering refined performance and cost.
Outcome:	The proposed model improves 8.2% from baseline and even with extremely small memory footprints, outperforms ChatGPT post-refinement.

Copied to clipboard

Andrei Catalin Coman, Christos Theodoropoulos, Marie-Francine Moens, James Henderson

Do Current Video LLMs Have Strong OCR Abilities? A Preliminary Study (2025.coling-main)

Challenge:	Existing methods that encode textual and structural information for inductive link prediction are frugal and fast at training and inference time.
Approach:	They propose a Transformer-based framework that unifies textual and structural information for inductive link prediction in text-attributed knowledge graphs by encoding ego-graphs (1-hop neighbourhoods).
Outcome:	The proposed framework can achieve superior performance on three popular datasets and reduce the reliance on resource-intensive encoders.

Copied to clipboard

Yulin Fei, Yuhui Gao, Xingyuan Xian, Xiaojin Zhang, Tao Wu, Wei Chen

Natural Language Processing for Achieving Sustainable Development: the Case of Neural Labelling to Enhance Community Profiling (2020.emnlp-main)

Challenge:	a new benchmark evaluates video-based optical character recognition (Video OCR) performance of multi-modal models in videos . the benchmark aims to improve video LLMs' ability to extract text from video content . previous benchmarks have focused on video QA, but not video-related QA.
Approach:	They propose to evaluate the video OCR performance of multi-modal models in videos . they use a semi-automated approach that integrates the OCR ability of image LLMs with manual refinement .
Outcome:	The proposed benchmark includes 1,028 videos and 2,961 question-answer pairs . it integrates the OCR ability of image LLMs with manual refinement .

Copied to clipboard

Costanza Conforti, Stephanie Hirmer, Dai Morgan, Marco Basaldella, Yau Ben Or

Evaluating the Consistency of LLM Evaluators (2025.coling-main)

Challenge:	In recent years, there has been an increasing interest in the application of Artificial Intelligence (AI) to the field of Sustainable Development (SD).
Approach:	They propose a new extreme multi-class multi-label Automatic UserPerceived Value classification task that uses a complex corpus of interviews to investigate the problem.
Outcome:	The proposed task solves a cost- and time-barrier in constructing qualitative data that prevents its widespread use and associated benefits.

Copied to clipboard

Noah Lee, Jiwoo Hong, James Thorne

Challenge:	Large language models (LLMs) have shown potential as general evaluators with the benefits of speed and cost.
Approach:	They conduct extensive studies on the two aspects of consistency in LLM evaluations, Self-Consistency (SC) and Inter-scale Consistency on different scoring scales and criterion granularity with open-source and proprietary models.
Outcome:	The results show that strong proprietary models are not necessarily consistent evaluators, highlighting the importance of considering consistency in assessing the capability of LLM evalueators.

IRT-Router: Effective and Interpretable Multi-LLM Routing via Item Response Theory (2025.acl-long)

Copied to clipboard

Wei Song, Zhenya Huang, Cheng Cheng, Weibo Gao, Bihan Xu, GuanHao Zhao, Fei Wang, Runze Wu

Linguistic Theories

Challenge:	Large language models have demonstrated exceptional performance across a wide range of tasks . however, selecting the optimal LLM to respond to a user query often necessitates a delicate balance between performance and cost.
Approach:	They propose a multi-LLM routing framework that efficiently routes user queries to the most suitable LLM.
Outcome:	The proposed framework outperforms baseline methods in terms of effectiveness and interpretability.

MDSEval: A Meta-Evaluation Benchmark for Multimodal Dialogue Summarization (2025.findings-emnlp)

Copied to clipboard

Yinhong Liu, Jianfeng He, Hang Su, Ruixue Lian, Yi Nian, Jake W. Vincent, Srikanth Vishnubhotla, Robinson Piramuthu, Saab Mansour

Dialogue Systems & Conversational Agents

Challenge:	Multimodal Dialogue Summarization (MDS) is a critical task with wide-ranging applications.
Approach:	They propose a meta-evaluation benchmark for multimodal dialogue summarization based on image-sharing dialogues, corresponding summaries and human judgments .
Outcome:	The proposed framework is the first to identify and formalize key evaluation dimensions specific to MDS.

Automatic Transmission for LLM Tiers: Optimizing Cost and Accuracy in Large Language Models (2025.findings-acl)

Copied to clipboard

Injae Na, Keonwoong Noh, Woohwan Jung

Large Language Models as Financial Data Annotators: A Study on Effectiveness and Efficiency (2024.lrec-main)

Challenge:	Large Language Models (LLMs) are powerful tools for a wide range of natural language tasks.
Approach:	They propose an LLM automatic transmission framework that automatically selects LLM tiers without training.
Outcome:	The proposed framework achieves superior performance while reducing costs.

Copied to clipboard

Toyin D. Aguda, Suchetha Siddagangappa, Elena Kochkina, Simerjot Kaur, Dongsheng Wang, Charese Smiley

Cost-Performance Optimization for Processing Low-Resource Language Tasks Using Commercial LLMs (2024.findings-emnlp)

Challenge:	Large Language Models (LLMs) have demonstrated remarkable performance in data annotation tasks on general domain datasets, but their effectiveness on domain specific datasets remains under-explored.
Approach:	They compare the annotations produced by three LLMs against expert annotators and crowdworkers.
Outcome:	The proposed models outperform expert crowdworkers and crowd-sourced annotators on domain specific datasets.

Copied to clipboard

Arijit Nag, Animesh Mukherjee, Niloy Ganguly, Soumen Chakrabarti

BiCSRouter: Bi-Level Cross-System Routing for Utility-Aware LLM Inference (2026.findings-acl)

Challenge:	Large Language Models (LLMs) exhibit impressive zero/few-shot inference and generation quality for high-resource languages (HRLs).
Approach:	They propose to reduce the cost of processing LRLs by code-mixing, translation, and transliteration of LRL to HRLs to ensure that predictive and generative qualities are not compromised.
Outcome:	The proposed model reduces the cost of processing LRLs while ensuring that predictive and generative qualities are not compromised.

Copied to clipboard

Mao Keyu, Eiki Murata, Ukyo Honda

Challenge:	Existing routing frameworks operate within a single computational paradigm . a cross-system routing framework that integrates two orthogonal regimes is proposed .
Approach:	They propose a cross-system routing framework that integrates two orthogonal regimes . they propose MBPP-based model that decomposes routing into intra-regime configuration selection and inter-regem system selection .
Outcome:	The proposed framework outperforms 15 representative baselines on MBPP and MATH benchmarks.

Cache me if you Can: an Online Cost-aware Teacher-Student framework to Reduce the Calls to Large Language Models (2023.findings-emnlp)

Copied to clipboard

Ilias Stogiannidis, Stavros Vassos, Prodromos Malakasiotis, Ion Androutsopoulos

NeuReasoner: Towards Explainable, Controllable, and Unified Reasoning via Mixture-of-Neurons (2026.acl-long)

Challenge:	Prompting Large Language Models (LLMs) performs impressively in zero- and few-shot settings.
Approach:	They propose a framework that allows reducing calls to LLMs by caching previous LLM responses and using them to train a local inexpensive model on the SME side.
Outcome:	The proposed framework reduces calls to LLMs by caching previous LLM responses and using them to train a local inexpensive model on the SME side.

Copied to clipboard

Haonan Dong, Kehan Jiang, Haoran Ye, Wenhao Zhu, Zhaolu Kang, Guojie Song

A Sequential Multi-Stage Approach for Code Vulnerability Detection via Confidence- and Collaboration-based Decision Making (2025.emnlp-main)

Challenge:	Existing Large Reasoning Models (LRMs) lack explainability and controllability . Existing models target isolated levels without unification, while relying on RL .
Approach:	They propose an explainable, controllable, and unified reasoning framework driven by MoN.
Outcome:	The proposed framework achieves performance gains of 27.0% while reducing token consumption by 19.6% 63.3%.

Copied to clipboard

Chung-Nan Tsai, Xin Wang, Cheng-Hsiung Lee, Ching-Sheng Lin

Challenge:	Large language models (LLMs) have shown strong capabilities across diverse domains, but their application to code vulnerability detection raises significant concerns regarding efficiency, scalability and cost.
Approach:	They propose a sequential multi-stage approach via confidence- and collaboration-based decision making via a three-stage sequential classification framework with a single agent, retrieval-augmented generation with external examples, and multi-agent reasoning enhanced with RAG.
Outcome:	The proposed approach improves code vulnerability detection performance on a benchmark dataset and a low-resource language.

GPT vs RETRO: Exploring the Intersection of Retrieval and Parameter-Efficient Fine-Tuning (2024.emnlp-main)

Copied to clipboard

Aleksander Ficek, Jiaqi Zeng, Oleksii Kuchaiev

CLIP-UP: A Simple and Efficient Mixture-of-Experts CLIP Training Recipe with Sparse Upcycling (2025.findings-emnlp)

Challenge:	Pre-trained large language models can be used for specific tasks and unique information but lack the resources for extensive retraining.
Approach:	They propose to use PEFT methods to adapt large language models while minimizing compute requirements.
Outcome:	The proposed methods outperform GPT models in zero-shot settings but lag behind PEFT.

Copied to clipboard

Xinze Wang, Chen Chen, Yinfei Yang, Hong-You Chen, Bowen Zhang, Aditya Pal, Xiangxin Zhu, Xianzhi Du

LangProBe: a Language Program Benchmark (2025.findings-emnlp)

Challenge:	Mixture-of-Experts (MoE) models are crucial for scaling model capacity while controlling inference costs.
Approach:	They propose an alternative training strategy that converts a dense CLIP model into a sparse MoE architecture.
Outcome:	The proposed training strategy outperforms dense models on COCO and Flickr30k benchmarks.

Copied to clipboard

Shangyin Tan, Lakshya A Agrawal, Arnav Singhvi, Liheng Lai, Michael J Ryan, Dan Klein, Omar Khattab, Koushik Sen, Matei Zaharia

Grounded Semantic Role Labelling from Synthetic Multimodal Data for Situated Robot Commands (2025.emnlp-main)

Challenge:	Composing language models into multi-step language programs is a mainstream paradigm for building AI systems, but tradeoffs in this space have only scarcely been studied before.
Approach:	They propose a benchmarking tool to evaluate the architectures and optimization strategies for language programs . they find that optimized language programs offer strong cost-quality Pareto improvement .
Outcome:	The proposed framework evaluates the impact of program architectures and optimizers on quality and cost.

Copied to clipboard

Claudiu Daniel Hromei, Antonio Scaiella, Danilo Croce, Roberto Basili

STAGE: Simple Text Data Augmentation by Graph Exploration (2024.lrec-main)

Challenge:	Existing symbolic parsers lack flexibility to operate in complex, dynamic environments.
Approach:	They propose a framework that combines frame semantics with perceptual grounding to enable robots to interpret commands via multimodal logical forms.
Outcome:	The proposed framework produces over 11,000 image-command pairs and lowers the cost of manual parsers.

Copied to clipboard

Ho-Seung Kim, YongHoon Kang, Jee-Hyong Lee

AgentSlimming: Towards Efficient and Cost-Aware Multi-Agent Systems (2026.acl-long)

Challenge:	Pre-trained language models (PLMs) are widely used for various tasks, but fine-tuning them requires sufficient data.
Approach:	They propose a method for data augmentation that utilizes a word-relation graph to select optimal words for each modification.
Outcome:	The proposed method is highly effective across diverse datasets and different PLMs.

Copied to clipboard

Yulang Chen, Haoxuan Peng, Jinyan Liu, Zichen Wen, Dongrui Liu, Linfeng Zhang

RouterHGC: Optimized Router for LLM-based Multi-Agent Systems via Heterogeneous Graph Contrastive Learning (2026.findings-acl)

Challenge:	Automated expansion methods often result in bloated structures with redundant agents, leading to excessive token consumption.
Approach:	They propose a plug-and-play compression framework for graph-structured multi-agent workflows . they estimate the importance score of each agent and remove redundant agents .
Outcome:	Experiments show that AgentSlimming reduces average token cost by 78.9% with negligible performance degradation.

Copied to clipboard

Yitao Xiao, Shaoyong Guo, Guoming Yang, Qingnan Wang, Yinlin Ren, Xuesong Qiu, Qi Feng

AOT*: Efficient Synthesis Planning via LLM-Empowered AND-OR Tree Search (2026.findings-acl)

Challenge:	Large Language Models (LLMs)-driven Multi-Agent Systems (MAS) have demonstrated remarkable scalability and generalizability across complex tasks.
Approach:	They propose a new framework for routing using large language models . they formalize routing as node selection through edge-weight prediction .
Outcome:	The proposed framework outperforms the best single LLM and baselines on five datasets . it achieves 0.80%–6.17% accuracy gains on MATH and HotpotQA while reducing inference cost by 27.40%.

Copied to clipboard

Xiaozhuang Song, Xuanhao Pan, Xinjian Zhao, Hangting Ye, Shufei Zhang, Jian Tang, Tianshu Yu