Papers by Chenlong Deng
RAG-Studio: Towards In-Domain Adaptation of Retrieval Augmented Generation Through Self-Alignment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing RAG systems that use pre-trained LLMs and retrievers often fail in specialized domains and applications. |
| Approach: | They propose a self-aligned training framework that adapts general RAG models to specific domains solely through synthetic data. |
| Outcome: | Experiments on specialized domain corpus, general LLM, and general retriever show that the self-aligned training framework outperforms human-annotated training data in specialized fields. |
An Element is Worth a Thousand Words: Enhancing Legal Case Retrieval by Incorporating Legal Elements (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for legal case retrieval lack the definition of relevance for legal cases . however, the definition goes beyond the common semantic relevance of ad-hoc retrieval. |
| Approach: | They propose a legal element dataset that incorporates legal elements into a semi-automatic method . they propose two models to enhance legal search using legal elements . |
| Outcome: | The proposed models outperform existing methods in enhancing legal search using legal elements. |
ChatRetriever: Adapting Large Language Models for Generalized and Robust Conversational Dense Retrieval (2024.emnlp-main)
Copied to clipboard
| Challenge: | a conversational search system requires accurate interpretation of user intent from complex multi-turn contexts. |
| Approach: | They propose a dual-learning approach that adapts LLMs for retrieval via contrastive learning while enhancing the complex session understanding through masked instruction tuning. |
| Outcome: | The proposed approach outperforms existing retrieval methods on five conversational search benchmarks. |
Learning Interpretable Legal Case Retrieval via Knowledge-Guided Case Reformulation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for legal case retrieval often overlook the incorporation of legal expert knowledge, leading to unsatisfactory retrieval performance. |
| Approach: | They propose a legal knowledge-guided case reformulation approach based on large language models for effective and interpretable legal case retrieval. |
| Outcome: | The proposed model performs better on complex legal case queries than existing methods. |
Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models (2025.acl-long)
Copied to clipboard
Zhisong Zhang, Yan Wang, Xinting Huang, Tianqing Fang, Hongming Zhang, Chenlong Deng, Shuaiyi Li, Dong Yu
| Challenge: | Large language models have demonstrated remarkable performance across a wide range of language tasks due to their remarkable ability in context modeling. |
| Approach: | They propose to use parallel context encoding to reduce attention entropy by incorporating attention sinks and selective mechanisms to reduce irregular attention . they also propose to incorporate attention sink mechanisms into the parallel encoded context to reduce the irregular attention. |
| Outcome: | The proposed methods lower irregular attention entropy and narrow performance gaps. |
Enabling Discriminative Reasoning in LLMs for Legal Judgment Prediction (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing large language models (LLMs) underperform in legal judgment prediction due to challenges in understanding case facts and distinguishing between similar charges. |
| Approach: | They propose a framework that allows LLMs to discriminate among charges and a judicial reasoning framework to improve their models for effective legal judgment prediction. |
| Outcome: | The proposed framework improves accuracy and efficiency when dealing with complex and confusing charges. |
GLARE: Agentic Reasoning for Legal Judgment Prediction (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models struggle with fine-grained distinctions between similar charges. |
| Approach: | They propose an agentic legal reasoning framework that actively retrieves external knowledge during decision-making. |
| Outcome: | The proposed model outperforms baseline models on complex cases involving confusing or rare charges on real-world datasets. |
A Silver Bullet or a Compromise for Full Attention? A Comprehensive Study of Gist Token-based Context Compression (2025.acl-long)
Copied to clipboard
| Challenge: | gist-based context compression methods can achieve only slight performance loss on tasks like retrieval-augmented generation and long-document QA, but it faces challenges in tasks like synthetic recall. |
| Approach: | They propose two strategies to improve gist-based context compression in large language models. |
| Outcome: | The proposed methods can achieve only slight performance loss on retrieval-augmented generation and long-document QA tasks, but they face challenges in tasks like synthetic recall. |