Papers by Dongjie Yang
PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to reduce memory usage for large language models neglect inter-layer dependency between layers and huge memory consumption in pre-computation. |
| Approach: | They propose a method that compresses the KV cache by layer-wise retaining crucial context. |
| Outcome: | The proposed method reduces memory usage by layer-wise retaining crucial context . it can improve 2.2x throughput compared to Accelerate with over 54% memory reduction . |
PACHAT: Persona-Aware Speech Assistant for Multi-party Dialogue (2025.emnlp-main)
Copied to clipboard
| Challenge: | Extensive research on spoken dialogue systems has advanced the development of intelligent voice assistants, but integration of role information within speech remains an underexplored area. |
| Approach: | They propose a language-based spoken dialogue system that integrates role information within speech to generate contextually appropriate responses. |
| Outcome: | The proposed architecture achieves speaker-specific responses, character understanding, and the generation of targeted replies in multi-party dialogue scenarios, surpassing existing spoken dialogue systems. |
Learning Better Masking for Better Language Model Pre-training (2023.acl-long)
Copied to clipboard
| Challenge: | Existing PrLMs adopt a Random-Token Masking strategy with a fixed masking ratio and different contents are masked by an equal probability throughout the training. |
| Approach: | They propose two scheduled masking approaches that adaptively tune masking ratio and masked content in different training stages, which improves pre-training efficiency and effectiveness. |
| Outcome: | The proposed methods improve the pre-training efficiency and effectiveness on the downstream tasks. |
Are LLMs Aware that Some Questions are not Open-ended? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown impressive capability to answer questions in a wide range of scenarios. |
| Approach: | They propose a method that enhances the question awareness of LLMs by adaptively adjusting the output distributions based on question features. |
| Outcome: | The proposed method improves the question awareness of LLMs by adaptively adjusting the output distributions based on question features. |
RefGPT: Dialogue Generation of GPT, by GPT, and for GPT (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have impressive capability to resolve a wide range of NLP tasks by fine-tuning high-quality instruction data. |
| Approach: | They propose a method to generate huge truthful and customized dialogues without worrying about factual errors caused by the model hallucination. |
| Outcome: | The proposed method solves the model hallucination in dialogue generation by restricting the LLMs to leverage the given reference instead of reciting their own knowledge to generate dialogues. |