Papers by Yutao Sun
Exploiting domain-slot related keywords description for Few-Shot Cross-Domain Dialogue State Tracking (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing frameworks for dialogue state tracking with domain-slot-value labels are expensive . current models are limited due to high cost of data annotation and lack of data in some domains . |
| Approach: | They propose a framework based on domain-slot related description to tackle the challenge of few-shot cross-domain DST. |
| Outcome: | The proposed framework outperforms existing methods on MultiWOZ and gains strong slot accuracy compared to existing models. |
The Self-Improvement Paradox: Can Language Models Bootstrap Reasoning Capabilities without External Scaffolding? (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to self-improvement rely on external supervision signals in the form of seed data and/or assistance from third-party models. |
| Approach: | They propose a framework for generating high-quality synthetic question-answer data in a fully autonomous manner. |
| Outcome: | The proposed framework generates high-quality synthetic question-answer data in a fully autonomous manner. |
Maximum Score Routing For Mixture-of-Experts (2025.findings-acl)
Copied to clipboard
| Challenge: | Traditional mixture-of-experts (MoE) networks impose an expert capacity constraint to ensure GPU-friendly computation. |
| Approach: | They propose a routing paradigm that dynamically allocates input tokens to top-k experts through differentiable sparse transformations, enabling scalable model capacity while preserving computational efficiency. |
| Outcome: | The proposed model achieves lower training losses and higher evaluation scores at equivalent FLOPs compared to constrained and unconstrained baselines. |
Joint Semantic and Strategy Matching for Persuasive Dialogue (2023.findings-emnlp)
Copied to clipboard
Chuhao Jin, Yutao Zhu, Lingzhen Kong, Shijie Li, Xiao Zhang, Ruihua Song, Xu Chen, Huan Chen, Yuchong Sun, Yu Chen, Jun Xu
| Challenge: | Persuasive dialogue models rely on utterance semantic matching and a key aspect has been ignored . compared with utterrance semantics, conversation strategies are high-level concepts, which can be informative and provide complementary information to achieve effective persuation. |
| Approach: | They propose to model conversation semantics and strategies to match them using a BERT-like module and an auto-regressive predictor. |
| Outcome: | The proposed model improves state-of-the-art by 5% on a small and 37% on 'large' datasets. |
Rectified Sparse Attention for Efficient Long-Sequence Generation (2026.findings-acl)
Copied to clipboard
Yutao Sun, Tianzhu Ye, Li Dong, Yuqing Xia, Jian Chen, Yizhao Gao, Shijie Cao, Jianyong Wang, Furu Wei
| Challenge: | Recent sparse decoding methods improve efficiency but suffer from KV cache misalignment, resulting in performance degradation. |
| Approach: | They propose a method that combines block-sparse attention with periodic dense rectification to bound error accumulation and preserve alignment with the pretraining distribution. |
| Outcome: | Experiments on math reasoning, language modeling, and retrieval tasks show that ReSA achieves near-lossless generation quality with significantly improved efficiency. |
ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability (2026.acl-long)
Copied to clipboard
| Challenge: | Existing rerankers perform poorly in complex ranking scenarios due to the scarcity of reasoning-intensive training data. |
| Approach: | They propose an automated reasoning-intensive training framework which generates high-quality training labels from training queries and passages. |
| Outcome: | The proposed model outperforms baselines significantly and achieves much lower latency than the pointwise reranker. |
Value type: the bridge to a better DST model (2023.findings-acl)
Copied to clipboard
| Challenge: | Value type of the slots can provide lots of useful information for DST tasks. however, it has been ignored in most previous works. |
| Approach: | They propose a new framework for DST task based on slot value type . they propose to extract the type of token from each turn and train a Ner model to extract corresponding type-entity from each conversation according to the token. |
| Outcome: | The proposed framework is effective on two multi-domain task-oriented conversation datasets. |
FocusLLM: Precise Understanding of Long Context by Dynamic Condensing (2025.acl-long)
Copied to clipboard
Zhenyu Li, Yike Zhang, Tengyu Pan, Yutao Sun, Zhichao Duan, Junjie Fang, Rong Han, Zixuan Wang, Jianyong Wang
| Challenge: | Existing context condensing methods cannot accurately understand the full context, as there is a considerable amount of information loss in the condensed process. |
| Approach: | They propose a framework to extend the fixed context length of any decoder-only LLM by distilling crucial information from long sequences. |
| Outcome: | The proposed framework extends the fixed context length of any decoder-only LLM, allowing it to focus on relevant information from very long sequences. |
Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers (2023.findings-acl)
Copied to clipboard
| Challenge: | Large pretrained language models have shown surprising in-context learning ability . despite the great success in performance, its working mechanism remains unclear . |
| Approach: | They explain language models as meta-optimizers and understand in-context learning as implicit finetuning . they find that Transformer attention has a dual form of gradient descent . |
| Outcome: | The proposed model can predict labels for unseen inputs without parameter updates . the proposed model outperforms smaller models with a single parameter update . |
Fine-Grained Legal Argument-Pair Extraction via Coarse-Grained Pre-training (2024.lrec-main)
Copied to clipboard
| Challenge: | Current methods conceptualize LAE as a supervised sentence-pair classification problem and necessitate extensive manual annotations. |
| Approach: | They propose a model that focuses on fine-grained alignment of argument pairs building upon coarse-grain complaint-defense pairs. |
| Outcome: | The proposed model outperforms baseline models by 3.7 and 2.4 points on average. |
A Length-Extrapolatable Transformer (2023.acl-long)
Copied to clipboard
Yutao Sun, Li Dong, Barun Patra, Shuming Ma, Shaohan Huang, Alon Benhaim, Vishrav Chaudhary, Xia Song, Furu Wei
| Challenge: | Existing Transformers can only deal with the in-distribution size of inputs. |
| Approach: | They propose a relative position embedding to explicitly maximize attention resolution . they also use blockwise causal attention during inference for better resolution a . |
| Outcome: | The proposed model achieves strong performance in interpolation and extrapolation settings. |