Papers by Jinyang Guo
Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling (2023.emnlp-main)
Copied to clipboard
| Challenge: | asymmetric outliers in transformer language models are a challenge for post-training quantization . we propose a framework for outlier suppression that can be seamlessly migrated into subsequent modules . |
| Approach: | They propose a framework for post-training quantization that includes the channel-wise shifting and scaling for concentration. |
| Outcome: | The proposed framework can be migrated into subsequent modules while maintaining equivalence. |
Dynamic Parallel Tree Search for Efficient LLM Reasoning (2025.acl-long)
Copied to clipboard
Yifu Ding, Wentao Jiang, Shunyu Liu, Yongcheng Jing, Jinyang Guo, Yingjie Wang, Jing Zhang, Zengmao Wang, Ziwei Liu, Bo Du, Xianglong Liu, Dacheng Tao
| Challenge: | Recent methods focus on search accuracy while overlooking computational efficiency. |
| Approach: | They propose a parallelism framework that dynamically optimizes reasoning path in inference. |
| Outcome: | The proposed framework improves efficiency by 2-4 on average while maintaining or even surpassing existing reasoning algorithms in accuracy. |
Context as a Tool: Context Management for Long-Horizon SWE-Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing large language models rely on append-only context maintenance or passively triggered compression heuristics, leading to context explosion, semantic drift, and degraded reasoning in long-running interactions. |
| Approach: | They propose a new context management paradigm that elevates context maintenance to a callable tool . they propose 'cat' framework that injects context-management actions into complete interaction trajectories . |
| Outcome: | The proposed model outperforms ReAct-based agents and static compression baselines on SWE-Verified tests. |
DB-LLM: Accurate Dual-Binarization for Efficient LLMs (2024.findings-acl)
Copied to clipboard
Hong Chen, Chengtao Lv, Liang Ding, Haotong Qin, Xiabin Zhou, Yifu Ding, Xuebo Liu, Min Zhang, Jinyang Guo, Xianglong Liu, Dacheng Tao
| Challenge: | Existing methods for ultra-low bit quantization cause severe accuracy drops . a novel Dual-Binarization method is proposed for efficient Large Language Models . |
| Approach: | They propose a Dual-Binarization method that takes 2-bit-width and binarization into account . they propose DB-LLM, which uses a 2-bit binarized weighted model to represent weights efficiently . |
| Outcome: | The proposed method surpasses the current State-of-the-Art in ultra-low bit quantization and achieves 20% reduction in computational consumption compared to the SOTA method under the same bit-width. |
LayoutMask: Enhance Text-Layout Interaction in Multi-modal Pre-training for Document Understanding (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained models on document images with transformer-based backbones have led to significant performance gains in this field. |
| Approach: | They propose a multi-modal pre-training model that combines text, layout and image . they propose to use local 1D position instead of global 1D positions as layout input . |
| Outcome: | The proposed model can achieve state-of-the-art results on a wide variety of VrDU problems. |
Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Prediction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in multimodal pre-trained models have significantly improved information extraction from visually-rich documents (VrDs). |
| Approach: | They propose a method to predict token sequences within visually-rich documents by a simple prediction head. |
| Outcome: | The proposed method can be used to predict token mentions as token sequences within documents. |
Adaptive Contrastive Knowledge Distillation for BERT Compression (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge distillation methods for BERT implicitly learn discriminative student features by mimicking the teacher features. |
| Approach: | They propose a new knowledge distillation approach called adaptive contrastive knowledge distilling for BERT compression using hidden state features in BERT as explicit supervision to learn discriminative student features. |
| Outcome: | The proposed approach improves on multiple natural language processing tasks. |
Half-S: Halving the Scale for Near-Lossless 4-Bit LLM Training (2026.findings-acl)
Copied to clipboard
Jinyang Du, Ruihao Gong, Linghan Ai, Zining Wang, Yunke Peng, Yao Wang, Lei Yan, null Wxuefei, Yaoyuan Wang, Jinyang Guo, Dahua Lin, Xianglong Liu
| Challenge: | Existing 4-bit training pipelines rely on max-scaling, which causes representation collapse . despite this, there are limitations in the accuracy of 4-bit LLM training . |
| Approach: | They propose a scaling strategy that uses half-scaling as a hardware-friendly default . they propose fp4 support that allows for a faster scaling of large language models . |
| Outcome: | The proposed scaling strategy narrows the gap between theoretical optimum and BF16 while maintaining the efficiency benefits of 4-bit training. |