Papers with performance
DL-QAT: Weight-Decomposed Low-Rank Quantization-Aware Training for Large Language Models (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Quantization-aware Training (QAT) is a popular technique for reducing memory usage and improving computational efficiency in large language models. |
| Approach: | They propose a weight-decomposed low-rank quantization-aware training approach that integrates QAT with a group-specific quantization magnitude adjustment. |
| Outcome: | The proposed method outperforms the state-of-the-art method on LLaMA and LLama2 models. |
Soft Self-Consistency Improves Language Models Agents (2024.acl-short)
Copied to clipboard
| Challenge: | Current “sample and select” methods rely on majority voting to score answers . however, when tasks have many distinct and valid answers, selection by voting requires a large number of samples. |
| Approach: | They introduce a method that replaces SC's discontinuous scoring with a continuous score computed from model likelihoods to increase selection even when actions are sparsely distributed. |
| Outcome: | The proposed method improves performance and efficiency on long-horizon interactive tasks by replacing SC’s discontinuous scoring with a continuous score computed from model likelihoods. |
Online Difficulty Filtering for Reasoning Oriented Reinforcement Learning (2026.eacl-long)
Copied to clipboard
| Challenge: | Recent advances in reinforcement learning with verifiable rewards (RLVR) show that large language models enhance their reasoning abilities when trained with veriable signals. |
| Approach: | They propose a method for a problem-aware filtering system that maximizes learning efficiency by selecting tasks of intermediate difficulty. |
| Outcome: | The proposed model improves when trained with verifiable rewards, but training efficiency is bottleneck . the proposed model achieves +12% gains in less than half the training steps of standard GRPO . |
Dense Template Retrieval for Customer Support (2022.coling-1)
Copied to clipboard
| Challenge: | Templated answers are used to cover a wide range of topics, but the number of templates is often too high for an agent to manually search. |
| Approach: | They propose a dense retrieval framework that adapts a standard in-batch negatives technique to support unpaired sampling of queries and templates. |
| Outcome: | The proposed approach improves performance and training speed over more standard methods. |
Unsupervised Anomaly Detection in Multi-Topic Short-Text Corpora (2023.eacl-main)
Copied to clipboard
| Challenge: | Unsupervised anomaly detection is a challenging task when the majority class is heterogeneous. |
| Approach: | They propose to use word embeddings to represent each sample by a dense vector and use a Mixture Model approach to detect which samples deviate the most from the underlying distributions of the corpus. |
| Outcome: | The proposed method is more efficient than state-of-the-art methods on real datasets. |
Efficient Convolutional Neural Networks for Diacritic Restoration (D19-1)
Copied to clipboard
| Challenge: | Diacritic restoration is a computational task that requires a computer to understand written texts. |
| Approach: | They propose to use Temporal Convolutional Neural Networks (TCN) to restore missing diacritics for each character in written text. |
| Outcome: | The proposed model improves on TCN in Arabic, Yoruba, and Vietnamese. |
Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysis (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies focus on analyzing structured data, while mining causal relationship among factors from unstructured data is of great importance. |
| Approach: | They propose a graph-based causal inference framework which builds causal graphs from fact descriptions without much human involvement. |
| Outcome: | The proposed framework can capture nuance from fact descriptions among confusing charges and provide explainable discrimination in few-shot settings. |
SEEKR: Selective Attention-Guided Knowledge Retention for Continual Learning of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods fail to fully exploit the knowledge embedded in models from previous tasks . Existing techniques fail to exploit the information embedded in previous tasks, resulting in a large number of replay samples to achieve good results. |
| Approach: | They propose a method that uses attention weights to extract knowledge from previous tasks . they use a data replay strategy to extract the knowledge from the previous tasks. |
| Outcome: | The proposed method achieves comparable or even better performance with only 1/10 of replayed data used by other methods. |
Topology-of-Question-Decomposition: Enhancing Large Language Models with Information Retrieval for Knowledge-Intensive Tasks (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are constrained to chaining immediate reasoning steps and relying solely on parametric knowledge. |
| Approach: | They propose a framework that activates retrieval only when necessary to improve answer accuracy. |
| Outcome: | Experiments show that the proposed framework improves performance in knowledge-intensive tasks. |
INFORM : Information eNtropy based multi-step reasoning FOR large language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated exceptional performance with dedicated Chain-of-Thought (CoT) prompts. |
| Approach: | They propose a new method by introducing information entropy as a criteria on for CoT prompt selection. |
| Outcome: | The proposed model outperforms existing models on seven reasoning benchmarks using two language models. |
Large Language Model as a Teacher for Zero-shot Tagging at Extreme Scales (2025.coling-main)
Copied to clipboard
| Challenge: | Extreme Zero-shot XMC uses lightweight bi-encoders to identify pseudo labels . state-of-the-art methods rely on suboptimal labels for training . |
| Approach: | They propose a framework that uses a lightweight bi-encoder to identify high-quality pseudo labels during training while employing a lightweight bi-coder for efficient inference. |
| Outcome: | The proposed framework achieves superior performance and efficiency over existing methods. |
GNNavi: Navigating the Information Flow in Large Language Models by Graph Neural Network (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit strong In-Context Learning (ICL) capabilities when prompts with demonstrations are used. |
| Approach: | They propose a prompt-based parameter-efficient fine-tuning approach that leverages insights into ICL’s information flow dynamics and hardwires the desired information flow into the GNN. |
| Outcome: | The proposed approach surpasses prompt-based fine-tuning methods in few-shot settings by updating just 0.2% to 0.5% of parameters. |
Investigating Efficiently Extending Transformers for Long Input Summarization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large pretrained Transformer models have proven capable at tackling natural language tasks, but handling long sequence inputs still poses a significant challenge. |
| Approach: | They propose an extension of the PEGASUS model with additional long input pretraining to handle inputs of up to 16K tokens. |
| Outcome: | The proposed model achieves strong performance on long input summarization tasks comparable with much larger models. |
Hierarchical Reinforcement Learning with Augmented Step-Level Transitions for LLM Agents (2026.acl-long)
Copied to clipboard
| Challenge: | Existing Large language model agents rely on increasingly long interaction histories, resulting in high computational cost and limited scalability. |
| Approach: | They propose a hierarchical reinforcement learning framework that enables step-level learning by conditioning only on single-step transitions rather than full interaction histories. |
| Outcome: | The proposed framework outperforms baselines on ScienceWorld and ALFWorld benchmarks in terms of performance and generalization while reducing token usage. |
Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex Reasoning (2025.findings-acl)
Copied to clipboard
| Challenge: | In-context learning (ICL) can enhance the complex reasoning capabilities of large language models. |
| Approach: | They propose a curriculum ICL strategy guided by problem-solving logic to select demonstration examples based on curriculum learning. |
| Outcome: | The proposed strategy outperforms existing methods on multiple benchmarks. |
SQATIN: Supervised Instruction Tuning Meets Question Answering for Improved Dialogue NLU (2024.naacl-long)
Copied to clipboard
| Challenge: | Task-oriented dialogue (TOD) systems support users in execution of specific, well-defined tasks through natural language interaction. |
| Approach: | They propose a framework for dialog NLU based on instruction tuning and question-answering-based formulation of ID and VE tasks. |
| Outcome: | The proposed framework surpasses existing models in training and cross-domain transfer and significantly outperforms existing large language models in performance and inference efficiency. |
Boosting Long-Context Information Seeking via Query-Guided Activation Refilling (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for processing long contexts are ineffective due to their inherent context window limitations and the computational burden of extensive key-value activations. |
| Approach: | They propose a method for processing long context information-seeking tasks via query-guided ACtivation REfilling (ACRE) a bi-layer KV Cache is constructed where the layer-1 cache compactly captures global information and the layer-2 cache provides detailed, localized information. |
| Outcome: | The proposed method achieves significant improvements in both performance and efficiency on a variety of long-context information-seeking datasets. |
FigMemes: A Dataset for Figurative Language Identification in Politically-Opinionated Memes (2022.emnlp-main)
Copied to clipboard
| Challenge: | FigMemes is a dataset for figurative language classification in politically-opinionated memes. |
| Approach: | They propose to use figurative language classification to identify politically-opinionated memes by analyzing their datasets and comparing them to other machine learning models. |
| Outcome: | The proposed dataset includes annotations of six commonly used types of figurative language in politically-opinionated memes and a wide range of topics and visual styles. |
Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Prompt tuning of Large Language Models (LLMs) can incur performance degradation or low training efficiency. |
| Approach: | They propose a prompt tuning approach with Adaptive Optimization to enable efficient FL of LLMs. |
| Outcome: | The proposed approach improves performance and efficiency simultaneously and addresses client drift problems on both the device and server sides. |
Flatness-Aware Prompt Selection Improves Accuracy and Sample Efficiency (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Manually "engineering" prompts for large language models can be laborious and time-intensive. |
| Approach: | They propose a new metric to quantify the expected utility of a language prompt. |
| Outcome: | The proposed metric outperforms previous prompt selection metrics with 10% increase in Pearson correlation across 6 classification benchmarks and the prompt selected by the proposed meter gains 5% higher accuracy than previous metrics. |
Uncertainty-aware Parameter-Efficient Self-training for Semi-supervised Language Understanding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for pre-trained language models rely on noisy data, which can be expensive if all parameters are updated. |
| Approach: | They propose a self-training framework that incorporates Monte Carlo dropouts into the model and judiciously selects reliable pseudo-labeled examples based on confidence and certainty. |
| Outcome: | The proposed framework improves performance and efficiency over multiple tasks over multiple datasets. |
Sequence Labeling Parsing by Learning across Representations (P19-1)
Copied to clipboard
| Challenge: | Constituency and dependency parsing are the main abstractions for representing syntactic structure of sentences . constituency parsers are considered disjointed tasks, and their improvements have been obtained separately. |
| Approach: | They propose to add auxiliary loss to constituency parsing paradigms and explore a model that parses both paradigms at no cost. |
| Outcome: | The proposed model outperforms single-task models by 1.05 F1 points and 0.62 UAS points for constituency parsing and dependency parsers. |
Language Detoxification with Attribute-Discriminative Latent Space (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to detoxify toxic text require excessive memory, computations and time. |
| Approach: | They propose a method to generate toxic text using an attribute-discriminative latent space. |
| Outcome: | The proposed method outperforms baselines on detoxified language and dialogue generation tasks while being time- and memory-efficient. |
𝜙-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time Exploration and Exploitation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing inference-time optimization strategies address the shortsightedness of auto-regressive generation, but the vast search space leads to excessive exploration and insufficient exploitation. |
| Approach: | They propose a decoding strategy that approximates two distributions via foresight and clustering to provide an efficient estimation of step value. |
| Outcome: | The proposed decoding strategy outperforms strong baselines in performance and efficiency. |
Is Multi-Hop Reasoning Really Explainable? Towards Benchmarking Reasoning Interpretability (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for multi-hop reasoning are not able to evaluate their interpretability . a recent study found that many paths are unreasonable . |
| Approach: | They propose a framework to evaluate the interpretability of multi-hop reasoning models . they annotate all possible rules and establish a benchmark . |
| Outcome: | The proposed framework outperforms existing models in terms of performance and interpretability. |
The Revolution of Multimodal Large Language Models: A Survey (2024.findings-acl)
Copied to clipboard
Davide Caffagni, Federico Cocchi, Luca Barsellotti, Nicholas Moratelli, Sara Sarto, Lorenzo Baraldi, Lorenzo Baraldi, Marcella Cornia, Rita Cucchiara
| Challenge: | Recent advances in large language models have led to the development of multimodal large language model. |
| Approach: | They present a review of recent visual-based Large Language Models and analyze their architectures and alignment strategies. |
| Outcome: | The proposed models can integrate visual and textual modalities while providing a dialogue-based interface and instruction-following capabilities. |
Nested-Refinement Metamorphosis: Reflective Evolution for Efficient Optimization of Networking Problems (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel in network algorithm design but suffer from inefficient iterative coding and high computational costs. |
| Approach: | They propose a method to iteratively refine task descriptions and metamorphosis on algorithms to generate more effective solutions. |
| Outcome: | Experimental results show that Nested-Refinement Metamorphosis outperforms state-of-the-art approaches in performance and efficiency. |
LongAttn: Selecting Long-context Training Data via Token-level Attention (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to select long-context data often rely on sentence-level analysis, which can be greatly optimized in both performance and efficiency. |
| Approach: | They propose a token-level framework which quantifies long-range dependencies for LLMs by calculating token-based dependency strength and distribution uniformity of token scores. |
| Outcome: | The proposed framework quantifies long-range dependencies, enabling more accurate and efficient data selection. |
Text-Guided Multi-Scale Frequency Representation Adaptation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning visual signals are limited by their size and complexity. |
| Approach: | They propose a multi-scale frequency-based fine-tuning method that integrates textual information and performs multi-level fine- tuning of visual signals in the frequency domain. |
| Outcome: | Extensive experiments on multimodal models, including CLIP and LLaVA, demonstrate that the proposed method significantly improves performance and efficiency with minimal cost and fast convergence within one epoch. |
JsonTuning: Towards Generalizable, Robust, and Controllable Instruction Tuning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing text-to-text methods struggle with issues such as generalization, robustness, and controllability due to their lack of explicit task structures. |
| Approach: | They propose a structure-to-structure approach that uses JSON structures to represent tasks. |
| Outcome: | The proposed method outperforms TextTuning in terms of performance, robustness, and controllability across different scenarios. |
Is Your Language Model Ready for Monetization Decisions? (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks focus on shopping-centric scenarios and user-facing data, overlooking intermediate decision stages and robustness considerations. |
| Approach: | They propose a multi-task benchmark to evaluate large language models in real-world monetization contexts. |
| Outcome: | The proposed benchmark covers intent understanding, commercial matching, and user behavior modeling. |