Distilling LLM Reasoning into Dense Encoders: Bridging the Accuracy-Efficiency Gap in Recommendation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing distillation approaches target Small Language Models (SLMs) or Conventional Recommendation Models, but face a critical trade-off between computational cost and semantic reasoning capacity. |
| Approach: | They propose a framework that establishes a text encoder as the optimal student architecture for scalable recommendation. |
| Outcome: | Experiments on four datasets show that the proposed framework outperforms state-of-the-art models and achieves significantly reduced latency. |
Similar Papers
SmartAD: Capacity-Aligned Agent Distillation for Small Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) show strong reasoning and decision-making ability, but their high inference cost motivates transferring agentic skills to small language models. |
| Approach: | They propose a capacity-aligned agent distillation framework that trains SLMs on full reason–act–observe trajectories from a tool-using teacher. |
| Outcome: | The proposed framework outperforms all baselines on multi-hop QA and math benchmarks with 1.5B and 3B models. |
Capturing Nuanced Preferences: Preference-Aligned Distillation for Small Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for aligning small language models with human values model preference knowledge from large language models (LLMs) however, this limitation hinders student SLMs from capturing nuanced preferences for multiple responses. |
| Approach: | They propose a framework which models teacher's preference knowledge as a probability distribution over all potential preferences, thereby providing more nuanced supervisory signals. |
| Outcome: | The proposed framework outperforms existing methods on four benchmark tasks and achieves 20% improvement on AlpacaEval 2 and Arena-Hard. |
Prompt Candidates, then Distill: A Teacher-Student Framework for LLM-driven Data Annotation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for data annotation use an aggressive approach prompting LLMs to determine a single gold label for each unlabeled sample. |
| Approach: | They propose a teacher-student framework that distills candidate annotations with a Small Language Model (SLM) they propose to use LLMs to generate and distill candidate annotation with slms to ensure unique labels are provided for downstream tasks. |
| Outcome: | The proposed method outperforms existing methods due to uncertainty in LLMs and is noisetolerant. |
Decoding in Latent Spaces for Efficient Inference in LLM-based Recommendation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Light Latent-space Decoding (L2D) is an efficient and efficient latent- space decoding method. |
| Approach: | They propose to bypass language-space decoding by matching candidate items with LLM's internal thought representations in the latent space. |
| Outcome: | The proposed method is 10x faster than language-space decoding while maintaining or enhancing performance. |
Rationale-Guided Distillation for E-Commerce Relevance Classification: Bridging Large Language Models and Lightweight Cross-Encoders (2025.coling-industry)
Copied to clipboard
| Challenge: | Large-scale e-commerce search systems typically follow a multi-step process to retrieve relevant products for a given query. |
| Approach: | They propose a distillation approach that uses "rationales" generated by Large Language Models to guide smaller cross-encoder models. |
| Outcome: | The proposed model achieves ROC-AUC improvements of 1.4% on 9 multilingual e-commerce datasets, 2.4% on 3 ESCI datasets and 6% on GLUE datasets while being 50 times faster per sample. |
Streamlining LLMs: Adaptive Knowledge Distillation for Tailored Language Models (2025.naacl-srw)
Copied to clipboard
| Challenge: | Large language models (LLMs) have transformative potential across industries, e.g., enhancing customer service, revolutionizing medical diagnostics, or identifying crises in news articles. |
| Approach: | They propose to distill compact, parameter-efficient tailored language models from LLMs for domain-specific tasks with comparable performance. |
| Outcome: | The proposed framework outperforms knowledge distillation frameworks in the crisis domain, where labeled data is scarce. |
Flipping Knowledge Distillation: Leveraging Small Models’ Expertise to Enhance LLMs in Text Matching (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities in acquiring diverse knowledge, making them highly effective across a wide range of tasks. |
| Approach: | They propose a flipped knowledge distillation paradigm where LLM learns from SLM . they propose to reinterpret LLMs as encoder-decoder models using LoRA . |
| Outcome: | The proposed model has been deployed in an online application environment and validated on financial and healthcare benchmarks and real-world applications. |
Sparse Logit Sampling: Accelerating Knowledge Distillation in LLMs (2025.acl-long)
Copied to clipboard
Anshumann Anshumann, Mohd Abbas Zaidi, Akhil Kedia, Jinwoo Ahn, Taehwak Kwon, Kangwook Lee, Haejun Lee, Joohyung Lee
| Challenge: | Knowledge distillation is a cost-effective technique to distill knowledge in Large Language Models, if the teacher output logits can be pre-computed and cached. |
| Approach: | They propose an importance-sampling-based method which provides unbiased estimates, preserves the gradient in expectation, and requires storing significantly sparser logits. |
| Outcome: | The proposed method enables faster training of student models with marginal overhead (10%) compared to cross-entropy based training, while maintaining competitive performance compared with full distillation. |
Generation-Distillation for Efficient Natural Language Understanding in Low-Data Settings (D19-61)
Copied to clipboard
| Challenge: | Recent research points to knowledge distillation as a potential solution for NLU tasks. |
| Approach: | They propose a training approach that distills large finetuned LMs into a small network using unlabeled training examples. |
| Outcome: | The proposed approach outperforms BERT training approaches while using 300 times fewer parameters. |
C2KD: Cross-layer and Cross-head Knowledge Distillation for Small Language Model-based Recommendation (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) show promise but their size and high inference costs limit deployment on resource-constrained devices. |
| Approach: | They propose a framework to transfer task-relevant knowledge from two complementary dimensions to Large Language Models (LLMs) Large Language models (LLMS) have demonstrated great potential in sequential recommendation tasks . |
| Outcome: | Extensive experiments across diverse model families show that the proposed framework achieves competitive performance compared to LLMs. |