Papers by Minyi Guo
Gumbel Reranking: Differentiable End-to-End Reranker Optimization (2025.acl-long)
Copied to clipboard
Siyuan Huang, Zhiyuan Ma, Jintao Du, Changhua Meng, Weiqiang Wang, Jingwen Leng, Minyi Guo, Zhouhan Lin
| Challenge: | Existing distillation-based approaches suffer from training-inference misalignment and fail to capture interdependencies among candidate documents. |
| Approach: | They propose a method to optimize rerankers by learning a stochastic, document-wise Top-k attention mask using the Gumbel Trick and Relaxed Top-K Sampling. |
| Outcome: | The proposed framework minimizes the overall language loss and improves recall on hotpotQA. |
On the (In-)Security of the Shuffling Defense in the Transformer Secure Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Existing work reveals only randomly permuted activations to the client, allowing adversaries to extract model weights. |
| Approach: | They propose an attack that aligns differently shuffled activations to a common permutation and exploits them to extract model weights. |
| Outcome: | The proposed attack can align shuffled activations to a common permutation and exploit them to extract model weights with a query cost of approximately $1. |
Transkimmer: Transformer Learns to Layer-wise Skim (2022.acl-long)
Copied to clipboard
| Challenge: | Prior work has proposed to augment Transformer model with the capability of skimming tokens to improve its computational efficiency. |
| Approach: | They propose to add a parameterized predictor before each layer that learns to make the skimming decision. |
| Outcome: | The proposed model achieves 10.97x speedup on GLUE benchmark compared with BERT-base baseline with less than 1% accuracy degradation. |
How Far Does BERT Look At: Distance-based Clustering and Analysis of BERT’s Attention (2020.coling-main)
Copied to clipboard
| Challenge: | Recent work on multi-head attention mechanism shows heuristics and clues in analyzing various aspects of the mechanism. |
| Approach: | They propose to cluster attention heatmaps into significantly different patterns through unsupervised clustering on top of a set of proposed features. |
| Outcome: | The proposed features can explain and calibrate different attention heads in Transformer models. |
CuBridge: An LLM-Based Framework for Understanding and Reconstructing High-Performance Attention Kernels (2026.acl-long)
Copied to clipboard
Xing Ma, Yangjie Zhou, Wu Sun, Zihan Liu, Jingwen Leng, Yun Lin, Shixuan Sun, Minyi Guo, Jin Song Dong
| Challenge: | Existing approaches to support diverse attention variants trade performance for flexibility . expert-written kernels achieve high efficiency but are difficult to adapt . |
| Approach: | They propose a framework that adapts expert-written attention kernels to GPUs . they use a structured lift–transfer–lower workflow to make execution explicit . |
| Outcome: | The proposed framework outperforms existing frameworks and compilers on diverse variants and GPU platforms. |