Towards Boosting LLMs-driven Relevance Modeling with Progressive Retrieved Behavior-augmented Prompting (2025.coling-industry)
Copied to clipboard
| Challenge: | Existing approaches to relevance modeling have lacked generalization and accuracy . recent studies have focused on capturing the semantic relationships between queries and items . |
| Approach: | They propose a framework that integrates world knowledge stored in LLMs with specialized domain knowledge represented by user behavior data for promising performance. |
| Outcome: | The proposed framework can handle full-scale search traffics of Alipay with acceptable cost and latency. |
Similar Papers
Efficient Integration of External Knowledge to LLM-based World Models via Retrieval-Augmented Generation and Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing attempts to enhance LLM-based world models through prompting or fine-tuning approaches are either requiring human knowledge or computationally extensive. |
| Approach: | They propose a framework that leverages retrieval-augmented generation to integrate external knowledge to LLM-based world models. |
| Outcome: | The proposed framework outperforms baseline models and exhibits strong generalizability. |
CPRM: A LLM-based Continual Pre-training Framework for Relevance Modeling in Commercial Search (2025.naacl-industry)
Copied to clipboard
Kaixin Wu, Yixin Ji, Zeyuan Chen, Qiang Wang, Cunxiang Wang, Hong Liu, Baijun Ji, Xu Jia, Zhongyi Liu, Jinjie Gu, Yuan Zhou, Linjian Mo
| Challenge: | Relevance modeling between queries and items is a key component of commercial search engines. |
| Approach: | They propose a framework for continual pre-training of LLMs to enhance domain knowledge . they employ queries and multi-field item to jointly pre-train for enhancing domain knowledge. |
| Outcome: | The proposed model achieves convincing performance compared to strong baselines. |
Leveraging Self-Attention for Input-Dependent Soft Prompting in LLMs (2025.acl-short)
Copied to clipboard
| Challenge: | Large language models require fine-tuning, which is computationally expensive and challenging. |
| Approach: | They propose a method that generates soft prompts based on input tokens and attends different tokens with varying importance. |
| Outcome: | The proposed method is simple and efficient, keeping the number of trainable parameters small. |
Systematic Knowledge Injection into Large Language Models via Diverse Augmentation for Domain-Specific RAG (2025.findings-naacl)
Copied to clipboard
Kushagra Bhushan, Yatin Nandwani, Dinesh Khandelwal, Sonam Gupta, Gaurav Pandey, Dinesh Raghu, Sachindra Joshi
| Challenge: | Retrieval-Augmented Generation (RAG) enhances response relevance by incorporating retrieved domain knowledge in the context, retrieval errors can still lead to hallucinations and incorrect answers. |
| Approach: | They propose a framework that augments the learning process by context augmentation and knowledge paraphrasing by incorporating retrieved domain knowledge into the context. |
| Outcome: | The proposed framework achieves 10% relative gain in token-level recall while preserving the LLM’s generalization capabilities. |
Dynamic Uncertainty Ranking: Enhancing Retrieval-Augmented In-Context Learning for Long-Tail Knowledge in LLMs (2025.naacl-long)
Copied to clipboard
| Challenge: | Prior work has shown that in-context learning (ICL) with retriever augmentation can help LLMs better capture long-tail knowledge, reducing their reliance on pre-trained data. |
| Approach: | They propose a reinforcement learning-based dynamic uncertainty ranking method that accounts for the varying impact of each retrieved sample on LLM predictions. |
| Outcome: | The proposed method outperforms baseline models on question-answering datasets by 2.76% and 5.96% on long-tail questions that elude zero-shot inference. |
Index-Time Prefix Injection for Multi-Tenant Retrieval: Improving Search Relevance Without Model Fine-Tuning (2026.acl-industry)
Copied to clipboard
| Challenge: | a single multilingual biencoder handles all retrieval, but these are task-generic and domain-agnostic. |
| Approach: | They propose a training-free method that prepending domain-descriptive prefixes to documents during indexing. |
| Outcome: | The proposed method improves retrieval relevance by prepending natural-language prefixes to documents during indexing. |
PepRec: Progressive Enhancement of Prompting for Recommendation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been gaining in-depth performance in natural language processing domains. |
| Approach: | They propose a training-free prompting framework that captures knowledge from content-based filtering and collaborative filtering to boost recommendation performance with LLMs. |
| Outcome: | The proposed framework outperforms traditional deep learning recommendation models and prompt-based recommendation systems on two real-world datasets. |
CuriousLLM: Elevating Multi-Document Question Answering with LLM-Enhanced Knowledge Graph Reasoning (2025.naacl-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have achieved significant success in open-domain question answering, however, they continue to face challenges such as knowledge cutoffs and hallucinations. |
| Approach: | They propose a new mechanism that integrates a curiosity-driven reasoning mechanism into an LLM agent to generate relevant follow-up questions. |
| Outcome: | The proposed enhancement integrates a curiosity-driven reasoning mechanism into an LLM agent, enabling it to generate relevant follow-up questions, thereby guiding the information retrieval process more efficiently. |
S3Prompt: Instructing the Model with Self-calibration, Self-recall and Self-aggregation to Improve In-context Learning (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models have limitations in practical applications, such as unsupervised generation and recall of in-context examples. |
| Approach: | They propose a self-calibration, self-recall and self-aggregation prompt pipeline to solve these problems. |
| Outcome: | The proposed pipeline improves the performance of large language models without annotating datasets and model parameter updates. |
ExPerT: Personalizing LLM Responses to Users’ Domain Expertise via Query-Wise Semantic and Keystroke Behavioral Cues (2026.acl-long)
Copied to clipboard
| Challenge: | Existing personalization methods relying on static profiles or text-only signals fail to capture query-specific expertise variation. |
| Approach: | They propose a query-wise personalization framework that adapts LLM responses to query domain expertise by combining semantic and behavioral cues. |
| Outcome: | ExPerT reduces expertise inference error by 65.7% compared to the strongest baseline and improves response satisfaction by 17.52% on a 5-point Likert scale. |