Papers by Guangxiang Zhao
LongAttn: Selecting Long-context Training Data via Token-level Attention (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to select long-context data often rely on sentence-level analysis, which can be greatly optimized in both performance and efficiency. |
| Approach: | They propose a token-level framework which quantifies long-range dependencies for LLMs by calculating token-based dependency strength and distribution uniformity of token scores. |
| Outcome: | The proposed framework quantifies long-range dependencies, enabling more accurate and efficient data selection. |
Thinking with Reasoning Skills: Fewer Tokens, More Accuracy (2026.acl-industry)
Copied to clipboard
| Challenge: | Reasoning LLMs often spend tokens on long intermediate reasoning traces when solving new problems. |
| Approach: | They propose to store reusable reasoning skills distilled from extensive deliberation and trial-and-error exploration and retrieve these skills at inference time to guide future reasoning. |
| Outcome: | The proposed approach reduces reasoning tokens while improving overall performance on coding and mathematical reasoning tasks. |
Review-Driven Multi-Label Music Style Classification by Exploiting Style Correlations (N19-1)
Copied to clipboard
| Challenge: | Several methods have been proposed for automatic music style classification, but they are limited in two aspects. |
| Approach: | They propose a deep learning approach to automatically learn and exploit style correlations by reviewing music reviews on websites. |
| Outcome: | The proposed approach performs well in capturing style correlations. |
Large Language Models Badly Generalize across Option Length, Problem Types, and Irrelevant Noun Replacements (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks have exposed patterns and may not truly assess generalization ability of Large Language Models (LLMs). |
| Approach: | They propose a “Generalization Stress Test” to assess Large Language Models’ generalization ability under slight and controlled perturbations, including option length, problem types, and irrelevant noun replacements. |
| Outcome: | The proposed test shows that LLMs exhibit severe accuracy drops and unexpected biases when faced with minor but content-preserving modifications. |
From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for natural language processing (NLP) are fine-tuned and released for research and deployments. |
| Approach: | They propose a PLM reuse paradigm that merges teacher-PLM knowledge into a student model. |
| Outcome: | The proposed paradigm can reduce the computational cost and environmental side-effects of retraining the PLM from scratch. |
Delving into the Openness of CLIP (2023.findings-acl)
Copied to clipboard
| Challenge: | Contrastive Language-Image Pre-training (CLIP) allows for open-vocabulary visual recognition, where the model can recognize images from an open class set in a zero-shot manner. |
| Approach: | They propose to use image classification as an image-to-text matching task instead of discrete category IDs to achieve open-vocabulary visual recognition. |
| Outcome: | The proposed model can recognize images from an open vocabulary in a zero-shot manner, but its performance deteriorates as the vocabulary expands. |
Learning Relation Alignment for Calibrated Cross-modal Retrieval (2021.acl-long)
Copied to clipboard
| Challenge: | despite advances in multimodal pre-training, cross-modal retrieval remains challenging . lack of relation consistency impairs contextualized representation of image-text pairs . |
| Approach: | They propose a new metric to quantify the relation consistency by measuring the semantic distance between linguistic and visual relations. |
| Outcome: | The proposed method boosts the performance of prevailing models on Flickr30k and MS COCO datasets by a considerable margin. |
Chain-of-Thought Matters: Improving Long-Context Language Models with Reasoning Path Supervision (2025.findings-emnlp)
Copied to clipboard
Dawei Zhu, Xiyu Wei, Guangxiang Zhao, Wenhao Wu, Haosheng Zou, Junfeng Ran, null XWang, Lin Sun, Xiangzheng Zhang, Sujian Li
| Challenge: | Recent advances in Large Language Models (LLMs) have highlighted the challenge of handling long-context tasks. |
| Approach: | They propose a chain-of-thought framework that teaches models to generate high-quality reasoning paths for enhanced long-context performance. |
| Outcome: | The proposed framework generalizes across most long-context scenarios and amplifys with increasing context length. |