Papers by Yau-Shian Wang
Attn-GS: Attention-Guided Context Compression for Efficient Personalized LLMs (2026.acl-long)
Copied to clipboard
Shenglai Zeng, Tianqi Zheng, Chuan Tian, Dante Everaert, Yau-Shian Wang, Yupin Huang, Michael J. Morais, Rohit Patki, Jinjin Tian, Xinnan Dai, Kai Guo, Monica Xiao Cheng, Hui Liu
| Challenge: | Existing approaches to personalize large language models (LLMs) rely on heuristic methods to compress user profiles but they ignore how LLMs process and prioritize different profile components. |
| Approach: | They propose an attention-guided context compression framework that leverages attention feedback from a marking model to mark important personalization sentences and guides a compression model to generate task-relevant compressed user contexts. |
| Outcome: | The proposed framework outperforms baselines across tasks, token limits, and settings while reducing token usage by 50 times. |
PESCO: Prompt-enhanced Self Contrastive Learning for Zero-shot Text Classification (2023.acl-long)
Copied to clipboard
| Challenge: | Existing text classification frameworks require large amounts of human-labeled documents to train . |
| Approach: | They propose a contrastive learning framework that improves zero-shot text classification . they add prompts to enhance label retrieval and use retrieved labels to enrich training . |
| Outcome: | The proposed framework achieves state-of-the-art on four benchmark text classification datasets. |
Generation-driven Contrastive Self-training for Zero-shot Text Classification with Instruction-following LLM (2024.eacl-long)
Copied to clipboard
| Challenge: | a novel method to train a smaller model with LLMs for zero-shot text classification requires immense computational resources due to their substantial model size. |
| Approach: | They propose a method which leverages the generative power of large language models to train a smaller model. |
| Outcome: | The proposed method outperforms state-of-the-art methods when limited data is available. |
Long-tailed Extreme Multi-label Text Classification by the Retrieval of Generated Pseudo Label Descriptions (2023.findings-eacl)
Copied to clipboard
| Challenge: | Extreme Multi-label text classification (XMTC) is a tough challenge due to the sheer size of the label spaces and the severe data scarcity problem associated with the long tail of rare labels in highly skewed distributions. |
| Approach: | They propose to use a trained bag-of-words classifier to generate pseudo label descriptions from a training bag- of-word classifier. |
| Outcome: | The proposed approach outperforms the existing models in the tail label prediction problem and achieves state-of-the-art (SOTA) performance on XMTC benchmark datasets. |