Papers by Zhewei Yao
Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQL (2026.findings-acl)
Copied to clipboard
Zhewei Yao, Guoheng Sun, Łukasz Borchmann, Zheyu Shen, Minghang Deng, Bohan Zhai, Hao Zhang, Ang Li, Yuxiong He
| Challenge: | Translating natural language questions into SQL is a core challenge in natural language understanding and human-computer interaction. |
| Approach: | They propose a reinforcement learning framework and model family to generate accurate, executable SQL using a lightweight reward signal based solely on execution correctness. |
| Outcome: | The proposed framework outperforms previous versions of 70B-class systems and achieves state-of-the-art execution accuracy across six diverse Text2SQL benchmarks. |
GRAD: Generalizing RAG Adaptation with Decoding (2026.acl-long)
Copied to clipboard
| Challenge: | Using GRAD, we can steer Retrieval-augmented generation objectives without retraining large language models. |
| Approach: | They propose an adaptive decoding-time framework that keeps the base generator fixed and composes small, objective-specific guidance at inference. |
| Outcome: | The proposed framework improves accuracy with favorable latency across public benchmarks and private settings with no in-domain labels while reliably activating helpful objectives and suppressing harmful ones, adaptively to tasks. |
PaCoRe: Learning to Scale Test-Time Compute with Parallel Coordinated Reasoning (2026.acl-long)
Copied to clipboard
Jingcheng Hu, Yinmin Zhang, Shijie Shang, Xiaobo Yang, Yue Peng, Zhewei Huang, Hebin Zhou, Xin Wu, Jie Cheng, Fanqi Wan, Xiangwen Kong, Chengyuan Yao, Kaiwen Yan, Ailin Huang, Hongyu Zhou, Qi Han, Zheng Ge, Xiangyu Zhang, Heung-Yeung Shum
| Challenge: | Parallel Coordinated Reasoning (PaCoRe) overcomes a central limitation of contemporary language models: their inability to scale test-time compute (TTC) far beyond sequential reasoning under a fixed context window. |
| Approach: | They propose a training-and-inference framework to overcome a central limitation of language models: their inability to scale test-time compute (TTC) under a fixed context window. |
| Outcome: | The proposed model scales to multi-million-token effective TTC without exceeding context limits. |
TAGQuant: Token-Aware Clustering for Group-Wise Quantization (2026.eacl-industry)
Copied to clipboard
| Challenge: | Existing work clusters channels using token dimension, which is suboptimal for grouping . a common challenge in LLM quantization is supporting "group-wise" quantization . |
| Approach: | They propose a method to group channels with similar activation distributions using tokens . they propose shuffle operation that reduces relative GSM8K error by 86% . |
| Outcome: | The proposed method reduces GSM8K error by 86% in both INT4 and MXFP4 formats compared to baselines . |
R3-SQL: Ranking Reward and Resampling for Text-to-SQL (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing rankers assign inconsistent scores to functionally equivalent SQL queries . ranking cannot recover when the correct SQL is absent from the pool. |
| Approach: | They propose a Text-to-SQL framework that rewards ranking and resampling . it first groups candidates by execution result and ranks groups for consistency . |
| Outcome: | The proposed framework achieves 75.03 execution accuracy on BIRD-dev, a new state of the art among methods using models with disclosed sizes. |
CORD: Balancing COnsistency and Rank Distillation for Robust Retrieval-Augmented Generation (2025.naacl-short)
Copied to clipboard
| Challenge: | Existing methods to ground large language models fail to adequately attend to all contexts . position bias is hindered by retrieval-augmented generation, which requires constant attention . |
| Approach: | They propose to augment and distill training instances with their perturbed positions to encourage consistent predictions . they also propose to balance COnsistency and Rank Distillation by combining noise-controlled perturbations with augmentation and distillation. |
| Outcome: | The proposed method outperforms existing methods in diverse RAG benchmarks. |
Optimizing Reasoning for Text-to-SQL with Execution Feedback (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models excel in many reasoning tasks, but their ability to leverage Chain-of-Thought (CoT) reasoning remains underexplored. |
| Approach: | They propose a framework that iteratively optimizes open-source LLMs by combining CoT reasoning with off-policy and on-poly DPO, relying solely on execution accuracy as feedback. |
| Outcome: | The proposed framework improves execution accuracy on BIRD and Spider datasets. |
Agentic Verification for Ambiguous Query Disambiguation (2026.findings-acl)
Copied to clipboard
Youngwon Lee, Seung-won Hwang, Ruofan Wu, Feng Yan, Danmei Xu, Moutasem Akkad, Zhewei Yao, Yuxiong He
| Challenge: | Prior Diversify-then-Verify pipelines generate interpretations and then retrieve evidence . ambiguous queries require RAG to disambiguate into interpretations that can be answered from corpus . |
| Approach: | They propose a novel approach that unifies diversification with verification by integrating retriever relevance and generator answerability feedback early. |
| Outcome: | The proposed approach improves grounding-aware F1 by 23% over baselines across multiple LLMs. |
What’s Hidden in a One-layer Randomly Weighted Transformer? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing subnetworks of one-layer randomly weighted neural networks can achieve impressive performance without changing initializations. |
| Approach: | They find subnetworks within one-layer randomly weighted neural networks that can achieve impressive performance without ever modifying the initializations. |
| Outcome: | The proposed subnetworks match 98%/92% of the performance of a trained Transformersmall/base on IWSLT14/WMT14. |
Scaling Vision-Language Models with Sparse Mixture of Experts (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a study explores the effectiveness of mixture-of-experts (MoE) techniques in scaling vision-language models . alayrac and colleagues demonstrate the effectiveness and performance of MoE in scaling VLMs . |
| Approach: | They propose to use sparsely-gated mixture-of-experts techniques to scale vision-language models . they show that MoE can achieve state-of the-art performance over dense models a range of benchmarks . |
| Outcome: | The proposed approach achieves state-of-the-art performance over dense models of equivalent computational cost. |
Inference Scaling for Bridging Retrieval and Augmented Generation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing work observed the generator bias, such that improving the retrieval results may negatively affect the outcome. |
| Approach: | They propose to use inference scaling to aggregate inference calls from the permuted order of retrieved contexts to create a new ranking. |
| Outcome: | The proposed approach improves ROUGE-L on MS MARCO and EM on HotpotQA benchmarks by 7 points. |
STUN: Structured-Then-Unstructured Pruning for Scalable MoE Pruning (2025.acl-long)
Copied to clipboard
| Challenge: | Mixture-of-experts (MoEs) have been adopted for reducing inference costs by sparsely activating experts in large language models (LLMs). |
| Approach: | They propose a structured-then-unstructured approach outperforming both of structured and unstructured pruning for MoEs. |
| Outcome: | The proposed approach outperforms both of structured and unstructured pruning, especially for MoEs with hundreds of experts. |
MAF: Multimodal Alignment Framework for Weakly-Supervised Phrase Grounding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on phrase localization uses caption-image datasets as weak supervision . existing work on supervised phrase localisation uses a large-scale annotated dataset . |
| Approach: | They develop a multimodal alignment framework to leverage more widely available caption-image datasets to model phrase relevance. |
| Outcome: | The proposed model improves on the widely-adopted Flickr30k dataset . it also improves the previous best unsupervised result by 5.56% . |
SwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model Transformation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are an integral enabler of enterprise applications such as summarization, retrieval augmented generation, and agentic workflows. |
| Approach: | They propose a model transformation and distillation procedure that prefills later layers’ KV cache using an earlier layer’s output, allowing prompt tokens to skip those later layers. |
| Outcome: | The proposed procedure can reduce prefill computation by 25-50% across several LLM families while incurring minimum quality degradation. |