Papers by Feiyu Xu
PositionID: LLMs can Control Lengths, Copy and Paste with Explicit Positional Awareness (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have impressive capabilities across various domains, including role-playing, creative writing, mathematical reasoning, and coding. |
| Approach: | They propose two methods to improve the model’s adherence to length constraints and copy-paste accuracy without compromising response quality. |
| Outcome: | The proposed methods improve the model’s adherence to length constraints and copy-paste accuracy without compromising response quality. |
FastMem: Fast Memorization of Prompt Improves Context Awareness of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models struggle with context awareness, leading to inaccuracies in tasks requiring faithful adherence to provided information. |
| Approach: | They propose a method to enhance LLMs' context awareness by updating only the last Feed-Forward Network module to maximize the likelihood of the prompt before inference . |
| Outcome: | The proposed method improves the accuracy of Llama 3-8B-Inst on the NQ-SWAP dataset from 59.1% to 71.6% and reduces the output structure failure rate of Qwen 1.5-4B-Chat from 34.9% to 25.5%. |
FRAME: Boosting LLMs with A Four-Quadrant Multi-Stage Pretraining Strategy (2025.findings-acl)
Copied to clipboard
Xuemiao Zhang, Feiyu Duan, Xu Liangyu, Yongwei Zhou, Sirui Wang, Rongxiang Weng, Jingang Wang, Xunliang Cai
| Challenge: | Multi-stage pretraining methods lack quantitative criteria for data partitioning and instead rely on intuitive heuristics. |
| Approach: | They propose a Four-quadRAnt Multi-stage prEtraining strategy that partitions data into four quadrants to achieve significant loss reductions four times. |
| Outcome: | The proposed strategy achieves 16.8% improvement over random across MMLU and CMMLU for the 3B model. |
DeMAC: Enhancing Multi-Agent Coordination with Dynamic DAG and Manager-Player Feedback (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Multi-agent systems (MAS) powered by large language models struggle to adapt to evolving task dependencies and to handle uncertainties. |
| Approach: | They propose a Dynamic Environment-Aware Manager-Player Agents Coordination framework that enhances multi-agent coordination through long-term strategic planning. |
| Outcome: | The proposed framework outperforms traditional reinforcement learning and human-agent collaboration in the Overcooked simulation. |
FIRE: Flexible Integration of Data Quality Ratings for Effective Pretraining (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to evaluate data quality rely on heuristic techniques or single quality signals. |
| Approach: | They propose a framework for integrating multiple data quality raters that integrates multiple quality signals into a unified space and provides a comprehensive quality signal for each data point. |
| Outcome: | The proposed framework outperforms existing methods and boosts model performance across a wide range of downstream tasks while requiring less than 37.5% tokens to reach the target performance. |
Cycle-Consistent Adversarial Autoencoders for Unsupervised Text Style Transfer (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for unsupervised text style transfer lack parallel data and difficulties in content preservation. |
| Approach: | They propose a neural approach to unsupervised text style transfer using non-parallel data. |
| Outcome: | The proposed approach can be trained end-to-end on two widely-used public datasets. |
Text2Mem: A Unified Memory Operation Language for Memory Operating System (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing memory frameworks lack a formal, executable specification for memory control. |
| Approach: | They propose a unified memory operation language that standardizes translation of natural-language instructions into reliable execution. |
| Outcome: | The proposed language standardizes translation of natural-language instructions into reliable execution. |
H-MAS: Hierarchical Multi-Agent Scheduling for Multi-Tenant LLM Serving (2026.findings-acl)
Copied to clipboard
Yuhan Liu, Cong Xu, Qi Jia, Yihua Wang, Feiyu Chen, Liang Jin, Lu Liu, Yaqian Zhao, Yuting Ding, Xiang Li
| Challenge: | Multi-tenant Model-as-a-Service (MaaS) workloads exhibit non-stationarity across multiple time scales . existing request schedulers often rely on a fixed policy that remains unchanged at runtime . |
| Approach: | They propose a hierarchical multi-agent scheduler that operates in a layered closed loop . they propose to maintain 1.2–3.0 higher Goodput than SGLang and vLLM . |
| Outcome: | Experiments show that H-MAS achieves 1.2–3.0 higher Goodput than SGLang and vLLM . it maintains more stable QoS under diverse request lengths and heterogeneous SLO targets . |
Preference Curriculum: LLMs Should Always Be Pretrained on Their Preferred Data (2025.findings-acl)
Copied to clipboard
Xuemiao Zhang, Xu Liangyu, Feiyu Duan, Yongwei Zhou, Sirui Wang, Rongxiang Weng, Jingang Wang, Xunliang Cai
| Challenge: | Existing methods of uniformly sampling data throughout the pretraining process are suboptimal because they overlook the model's evolving data preferences. |
| Approach: | They propose a Perplexity Difference (PD) based Preference Curriculum learning framework which perceives and uses the data preferred by LLMs as their capabilities improve . they propose PDPC to complete the arrangement of the dataset offline and ensure continuous training without interruption. |
| Outcome: | The proposed framework surpasses baselines on 1.3B and 3B models and achieves an increased average accuracy of over 8.1% across MMLU and CMMLU. |
Inside Out: Evolving User-Centric Core Memory Trees for Long-Term Personalized Dialogue Systems (2026.acl-long)
Copied to clipboard
| Challenge: | Existing personalized dialogue systems struggle to reconcile unbounded interactions with finite context constraints. |
| Approach: | They propose a framework that utilizes a globally maintained PersonaTree as the carrier of long-term user profiling. |
| Outcome: | The proposed framework outperforms existing systems in suppressing contextual noise and persona inconsistency. |