Papers by Lili Jiang
LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly lengthy and require longer prompts . this paper presents a coarse-to-fine prompt compression method to reduce cost and increase performance. |
| Approach: | They propose a coarse-to-fine prompt compression method that maintains semantic integrity under high compression ratios and a token-level iterative compression algorithm to better model the interdependence between compressed contents. |
| Outcome: | The proposed method yields state-of-the-art performance and allows for up to 20x compression with little performance loss over four datasets from different scenarios. |
Mitigate Position Bias in LLMs via Scaling a Single Hidden States Channel (2025.findings-acl)
Copied to clipboard
Yijiong Yu, Huiqiang Jiang, Xufang Luo, Qianhui Wu, Chin-Yew Lin, Dongsheng Li, Yuqing Yang, Yongfeng Huang, Lili Qiu
| Challenge: | Long-context language models exhibit position bias, also known as "lost in the middle" research shows that even long-contemporary LLMs fail to utilize all context information effectively . |
| Approach: | They propose a method to mitigate position bias by scaling positional hidden states . they propose to use a channel of hidden states to modify positional Hidden states a LCLM's positional bias . |
| Outcome: | The proposed method can improve performance by 15.2% in a "lost in the middle" benchmark. |
LeanK: Learnable K Cache Channel Pruning for Efficient Decoding (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing efforts to optimize the key-value (KV) cache include: (1) Eviction, which discards cache of less important tokens; (2) Selection, which retains the full KV cache but selectively reads relevant entries. |
| Approach: | They propose a learning-based method that prunes unimportant key (K) cache channels by leveraging static channel sparsity. |
| Outcome: | Experiments show that LeanK reduces GPU memory and accelerates decoding without sacrificing accuracy. |
Multimodal Review Generation with Privacy and Fairness Awareness (2020.coling-main)
Copied to clipboard
| Challenge: | Existing frameworks for generating personalized reviews take privacy and fairness into account . users generate digital footprints when "traveling" on the internet . |
| Approach: | They propose a neural-based framework that generates personalized reviews with privacy and fairness in mind. |
| Outcome: | The proposed framework generates plausibly long reviews while controlling the amount of exploited user data and using the least sentiment biased embeddings. |
Accelerating Prefilling via Decoding-time Contribution Sparsity (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing acceleration methods exploit attention score sparsity by estimating blocks with high attention scores and applying dynamic sparse attention. |
| Approach: | They propose a method which replaces dense attention with Triangle attention in a subset of layers to reduce the time needed to decode. |
| Outcome: | Experiments show that TriangleMix achieves near-lossless performance on long-context and long-constrast reasoning benchmarks while significantly improving efficiency. |
Position Engineering: Boosting Large Language Models through Positional Information Manipulation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have demonstrated significant strides towards achieving artificial general intelligence. |
| Approach: | They propose a technique termed position engineering which alters the positional information in the prompt without modifying the text itself. |
| Outcome: | The proposed technique significantly improves on the baseline in retrieval-augmented generation and in-context learning scenarios. |
LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression (2024.findings-acl)
Copied to clipboard
Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, Dongmei Zhang
| Challenge: | Existing approaches to compress prompts only leverage unidirectional context, causing suboptimal results. |
| Approach: | They propose a task-agnostic prompt compression method that takes tokens from context . they use a Transformer encoder to capture all essential information needed for prompt compression . |
| Outcome: | The proposed method is 3x-6x faster than existing prompt compression methods and faster than baselines. |
RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimization (2026.acl-long)
Copied to clipboard
Yihong Dong, Xue Jiang, Yongding Tao, Huanyu Liu, Kechi Zhang, Lili Mou, Rongyu Cao, Yingwei MA, Jue Chen, Binhua Li, Zhi Jin, Fei Huang, Yongbin Li, Ge Li
| Challenge: | Reinforcement Learning with Verifiable Reward (RLVR) has significantly advanced the complex reasoning abilities of Large Language Models (LLMs). |
| Approach: | They propose a hybrid-policy optimization approach that synergizes internal exploitation with external data to achieve stronger reasoning capabilities. |
| Outcome: | The proposed approach achieves state-of-the-art performance on six math reasoning benchmarks and superior performance on out-of distribution reasoning tasks. |
LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression (2024.acl-long)
Copied to clipboard
| Challenge: | Longer prompts introduce irrelevant and redundant information, which can weaken LLMs' performance. |
| Approach: | They propose a prompt compression tool that improves LLMs' perception of key information in input prompts by up to 21.4% with around 4x fewer tokens in GPT-3.5-Turbo. |
| Outcome: | The proposed solution improves performance and reduces costs and latency by up to 21.4% with around 4x fewer tokens in the NaturalQuestions benchmark. |