Not All Options Are Created Equal: Textual Option Weighting for Token-Efficient LLM-Based Knowledge Tracing (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have strong reasoning and generalization abilities, but they struggle to reflect the histories of example learners within a single prompt during in-context learning. |
| Approach: | They propose a LLM-based option weighted knowledge tracing framework that encodes the interaction histories of example learners in context as textual categorical option weights. |
| Outcome: | The proposed framework outperforms existing LLM-based KT models in warm-start and few-shot settings. |
Similar Papers
LLM-KT: Enhancing Large Language Models with Knowledge Tracing via Multi-Level Plug-and-Play Alignment (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to learn behavioral sequences fail to capture complex behavioral patterns due to a lack of deep reasoning capabilities and world knowledge. |
| Approach: | They propose a framework that integrates the reasoning power of Large Language Models with the sequential modeling strengths of traditional KT methods via multi-level plug-and-play alignment. |
| Outcome: | Extensive experiments on four standard datasets show that the proposed framework outperforms existing methods on state-of-the-art questions. |
Token Weighting for Long-Range Language Modeling (2025.findings-naacl)
Copied to clipboard
| Challenge: | Many applications of large language models (LLMs) require long-context understanding, but models still struggle with such tasks. |
| Approach: | They propose token-weighting schemes that assign different weights to each training token in the loss, generalizing existing works. |
| Outcome: | The proposed methods compare confidences of a long-context and short-concept model and show that non-uniform loss weights improve the long-constability of LLMs. |
LLMEmbed: Rethinking Lightweight LLM’s Genuine Function in Text Classification (2024.acl-long)
Copied to clipboard
| Challenge: | Recent attempts to improve text classification performance are based on heuristic Chain-of-Thought (CoT) LLMEmbed is a simple and effective transfer learning strategy that can be used to improve the performance of large language models. |
| Approach: | They propose a simple transfer learning strategy to improve text classification using heuristic Chain-of-Thought. |
| Outcome: | The proposed method achieves strong performance on publicly available datasets while using low training overhead. |
Improving the OOD Performance of Closed-Source LLMs on NLI Through Strategic Data Selection (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to improve robustness require changing the fine-tuning process or large-scale data augmentation, which are infeasible or cost prohibitive for closed-source models. |
| Approach: | They propose to prioritize more complex examples or replace existing training examples with LLM-generated data to improve performance on OOD NLI datasets. |
| Outcome: | The proposed methods improve performance on difficult OOD datasets while training with synthetic data leads to substantial improvements on easier OOD data. |
Faster In-Context Learning for LLMs via N-Gram Trie Speculative Decoding (2025.emnlp-main)
Copied to clipboard
| Challenge: | In-Context Learning (ICL) is a key method in prompt engineering, but its long retrieved contexts and limited token throughput will slow reasoning speeds. |
| Approach: | They propose a method that leverages the overlap between context and model output to generate drafts from the context. |
| Outcome: | The proposed method achieves the highest mean speedup on Vicuna-7B, Llama2-7B-Chat, and Llma3-8B-Instruct tasks. |
LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly lengthy and require longer prompts . this paper presents a coarse-to-fine prompt compression method to reduce cost and increase performance. |
| Approach: | They propose a coarse-to-fine prompt compression method that maintains semantic integrity under high compression ratios and a token-level iterative compression algorithm to better model the interdependence between compressed contents. |
| Outcome: | The proposed method yields state-of-the-art performance and allows for up to 20x compression with little performance loss over four datasets from different scenarios. |
GPT-Fathom: Benchmarking Large Language Models to Decipher the Evolutionary Path towards GPT-4 and Beyond (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing LLM leaderboards often reference scores reported in other papers without consistent settings and prompts, which may encourage cherry-picking favored settings and for better results. |
| Approach: | They propose an open-source and reproducible LLM evaluation suite built on top of OpenAI Evals that systematically evaluates 10+ leading LLMs and OpenAI’s legacy models on 20+ curated benchmarks across 7 capability categories. |
| Outcome: | The evaluation suite is built on top of OpenAI Evals and evaluates 10+ leading LLMs and OpenAI’s legacy models on 20+ curated benchmarks across 7 capability categories. |
Towards Context-Robust LLMs: A Gated Representation Fine-tuning Approach (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) enhanced with external contexts face challenges in handling imperfect evidence. |
| Approach: | They propose a framework that can balance internal knowledge with external contexts . they propose gating mechanisms and low-rank representation adapters to adjust hidden representations based on a lightweight intervention function . |
| Outcome: | The proposed model can effectively balance internal knowledge with external context, similar to human cognitive processes. |
LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression (2024.acl-long)
Copied to clipboard
| Challenge: | Longer prompts introduce irrelevant and redundant information, which can weaken LLMs' performance. |
| Approach: | They propose a prompt compression tool that improves LLMs' perception of key information in input prompts by up to 21.4% with around 4x fewer tokens in GPT-3.5-Turbo. |
| Outcome: | The proposed solution improves performance and reduces costs and latency by up to 21.4% with around 4x fewer tokens in the NaturalQuestions benchmark. |
PrExMe! Large Scale Prompt Exploration of Open Source LLMs for Machine Translation and Summarization Evaluation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are useful for low-resource scenarios and time-restricted applications. |
| Approach: | They propose a large-scale evaluation tool for large language models that uses prompts . they evaluate 720 prompt templates for open-source LLM-based metrics on MT and summarization datasets a 6.6M evaluations. |
| Outcome: | The proposed model evaluates 720 prompt templates on machine translation and summarization datasets. |