Papers by Yao Qian
RSMeM: Knowledge-Enhanced Memory Evolution for Remote Sensing Agents with Systematic Evaluation (2026.acl-long)
Copied to clipboard
Bingxian Wu, Yu Zhang, Zonghao Guo, Tang Liu, Chen Qian, Yuxiang Lu, Xingbo Du, Yanghao Li, Yidan Zhang, Chi Chen, Ling Yao, Chenghu Zhou, Maosong Sun
| Challenge: | Existing RS agents built on general-purpose LLMs are domain-agnostic, resulting in brittle and error-prone workflows. |
| Approach: | They propose a knowledge-enhanced memory evolution mechanism that bootstraps RS agents with pre-distilled domain knowledge and iteratively integrates online experience for robust multi-step tool execution. |
| Outcome: | Experiments show that the new model improves tool-use performance and accuracy . iteratively, iteration of the model integrates online experience for robust multi-step tool execution . |
SelfRACG: Enabling LLMs to Self-Express and Retrieve for Code Generation (2025.emnlp-main)
Copied to clipboard
Qian Dong, Jia Chen, Qingyao Ai, Hongning Wang, Haitao Li, null Yiwu, Yao Hu, Yiqun Liu, Shaoping Ma
| Challenge: | Existing retrieval-augmented code generation methods fail to accurately fetch the knowledge required for code generation for consecutive code fragments. |
| Approach: | They propose a paradigm that enables large language models to Self-express their information needs to enhance retrieval-augmented code generation methods. |
| Outcome: | Experiments show that SelfRACG can retrieve external knowledge that better aligns with the LLM’s own information needs, resulting in superior generation performance compared to vanilla RACG. |
CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents (2025.findings-acl)
Copied to clipboard
Tianqi Xu, Linyao Chen, Dai-Jie Wu, Yanjun Chen, Zecheng Zhang, Xiang Yao, Zhiqiang Xie, Yongchao Chen, Shilong Liu, Bochen Qian, Anjie Yang, Zhaoxuan Jin, Jianbo Deng, Philip Torr, Bernard Ghanem, Guohao Li
| Challenge: | Existing benchmarks for MLM agents in interactive environments are limited by their focus on a single environment, lack of detailed and generalized evaluation methods, and the complexity of constructing tasks and evaluators. |
| Approach: | They propose a cross-environment agent benchmark framework that integrates graph-based evaluation and task generation methods. |
| Outcome: | The proposed framework supports multiple devices and can be easily extended to any environment with a Python interface. |
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data (2024.findings-naacl)
Copied to clipboard
Ziyi Yang, Mahmoud Khademi, Yichong Xu, Reid Pryzant, Yuwei Fang, Chenguang Zhu, Dongdong Chen, Yao Qian, Xuemei Gao, Yi-Ling Chen, Robert Gmyr, Naoyuki Kanda, Noel Codella, Bin Xiao, Yu Shi, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang
| Challenge: | i-Code V2 is one of the first models capable of generating natural language from any combination of Vision, Language, and Speech data. |
| Approach: | They propose to create a model that can generate natural language from any combination of Vision, Language, and Speech data. |
| Outcome: | i-Code V2 matches or outperforms state-of-the-art single- and dual-modality baselines on 7 multimodal tasks. |
RealBench: A Chinese Multi-image Understanding Benchmark Close to Real-world Scenarios (2025.findings-emnlp)
Copied to clipboard
Fei Zhao, Chengqiang Lu, Yufan Shen, Qimeng Wang, Yicheng Qian, Haoxin Zhang, Yan Gao, null Yiwu, Yao Hu, Zhen Wu, Shangyu Xing, Xinyu Dai
| Challenge: | RealBench is the first Chinese multimodal multi-image dataset . the dataset contains 9393 samples and 69910 images . |
| Approach: | They propose to create a Chinese multimodal multi-image dataset using 21 models . they use closed-source models that support multi-inputs as well as open-source visual and video models a . |
| Outcome: | The first Chinese multimodal multi-image dataset contains 9393 samples and 69910 images. |
Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on pre-trained LLMs to better understand and improve their trustworthiness. |
| Approach: | They apply linear probing to LLMs to explore five key dimensions of trustworthiness: reliability, privacy, toxicity, fairness, and robustness. |
| Outcome: | The proposed model can distinguish concepts in each trustworthiness dimension, suggesting that it can be trained in early pre-training. |
EscapeBench: Towards Advancing Creative Intelligence of Language Model Agents (2025.acl-long)
Copied to clipboard
Cheng Qian, Peixuan Han, Qinyu Luo, Bingxiang He, Xiusi Chen, Yuji Zhang, Hongyi Du, Jiarui Yao, Xiaocheng Yang, Denghui Zhang, Yunzhu Li, Heng Ji
| Challenge: | Existing language model agents excel in planning and reasoning, but lack creativity in unfamiliar environments. |
| Approach: | They propose a benchmark suite of room escape game environments to challenge agents with creative reasoning, unconventional tool use and iterative problem-solving to uncover implicit goals. |
| Outcome: | The proposed framework can perform with 40% fewer steps and hints and performs robustly across difficulty levels. |
PEC-Home: Interpretation of Progressively Elliptical Commands in Smart Homes (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing home assistants struggle to interpret elliptical commands based on ellipine expressions . current assistants overlook the progressive omission that occurs in human dialogue as context accumulates - limiting their effectiveness in real-world applications . |
| Approach: | They propose a simulated home dataset specifically designed for interpreting progressively elliptical commands in smart homes. |
| Outcome: | The proposed dataset shows that existing home assistants struggle to execute user-intended operations based solely on elliptical commands. |
EcoLANG: Efficient and Effective Agent Communication Language Induction for Social Simulation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated impressive ability to role-play humans and replicate complex social dynamics. |
| Approach: | They propose an efficient agent communication language induction for social simulations that reduces token consumption by over 20%. |
| Outcome: | The proposed model reduces token consumption by over 20% while preserving human language. |
SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation (2026.findings-acl)
Copied to clipboard
Shuang Cheng, Yihan Bian, Dawei Liu, Yuhua Jiang, Yihao Liu, Linfeng Zhang, Qian Yao, Zhongbo Tian, Wenhai Wang, Qipeng Guo, Kai Chen, Biqing Qi, Bowen Zhou
| Challenge: | Autoregressive (AR) language models are a dominant paradigm in the field of parallelism and non-causal modeling. |
| Approach: | They propose a blockwise discrete diffusion model that preserves AR-compatible serving while enabling parallel intra-block generation. |
| Outcome: | The proposed model achieves theoretical speedups over 5 and wall-clock speedup of 2.3 on H200 GPUs in latency-critical regimes. |
Audio-Aware Large Language Models as Judges for Speaking Styles (2025.findings-emnlp)
Copied to clipboard
Cheng-Han Chiang, Xiaofei Wang, Chung-Ching Lin, Kevin Lin, Linjie Li, Radu Kopetz, Yao Qian, Zhendong Wang, Zhengyuan Yang, Hung-yi Lee, Lijuan Wang
| Challenge: | Audio-aware large language models (ALLMs) can understand textual and non-textual information in the audio input. |
| Approach: | They use audio-aware large language models (ALLMs) to evaluate the speaking styles of SLMs on two tasks: voice style instruction following and role-playing. |
| Outcome: | The proposed models can understand the textual and non-textual information in the audio input and can be used as a judge to assess the speaking styles of SLMs. |
Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Study (2025.coling-main)
Copied to clipboard
Bowen Li, Wenhan Wu, Ziwei Tang, Lin Shi, John Yang, Jinyang Li, Shunyu Yao, Chen Qian, Binyuan Hui, Qicheng Zhang, Zhiyin Yu, He Du, Ping Yang, Dahua Lin, Chao Peng, Kai Chen
| Challenge: | Existing benchmarks focused on simplified or isolated aspects of coding, ignoring the full spectrum of programming challenges. |
| Approach: | They propose a case study that examines the performance of large language models across the entire software development lifecycle with four programming languages, multiple domains, and carefully designed and verified metrics for each task. |
| Outcome: | The proposed model performs across the entire software development lifecycle, including design, environment setup, implementation, acceptance testing, and unit testing. |
Joint Pre-Encoding Representation and Structure Embedding for Efficient and Low-Resource Knowledge Graph Completion (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge graph completion models require longer training and inference times as well as increased memory usage. |
| Approach: | They propose to encode textual descriptions into semantic representations before training and integrate structural embedding with pre-encoded semantic description to improve model's prediction performance on 1-N relations. |
| Outcome: | The proposed model increases inference speed by 30x and reduces training memory by approximately 60% on the WN18RR and UMLS datasets. |
Ground Then Rank: Revisiting Knowledge-Based VQA with Training-Free Entity Identification (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing multi-modal retrieval augmented generation (MM-RAG) methods tightly couple entity discrimination and section-level evidence ranking into a single re-ranking stage, leading to high cost and limited generalization. |
| Approach: | They propose a framework that decouples entity identification from section-level re-ranking. |
| Outcome: | The proposed framework outperforms fine-tuned multi-modal re-ranking baselines while reducing training and inference complexity. |
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing (2022.acl-long)
Copied to clipboard
Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, Furu Wei
| Challenge: | Existing work shows that pre-trained models can improve in various natural language processing tasks. |
| Approach: | They propose a unified-modal encoder-decoder framework that pre-trains speech-text representations using large-scale unlabeled speech and text data. |
| Outcome: | The proposed framework is superior to existing models on speech-to-text processing tasks. |
Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization (2025.emnlp-main)
Copied to clipboard
Shuo Xing, Peiran Li, Yuping Wang, Ruizheng Bai, Yueqi Wang, Chan-Wei Hu, Chengxuan Qian, Huaxiu Yao, Zhengzhong Tu
| Challenge: | emergence of large Vision Language Models (VLMs) has broadened the capabilities of single-modal Large Language Model (LLM) but VLMs are prone to significant hallucinations, especially in the form of cross-modal inconsistencies. |
| Approach: | They propose a new alignment framework that leverages image retrieval to integrate both textual and visual preference signals. |
| Outcome: | The proposed framework mitigates hallucinations more effectively than previous methods . it maintains robustness and scalability across a wide range of VLM sizes and architectures . |
i-Code Studio: A Configurable and Composable Framework for Integrative AI (2024.emnlp-demo)
Copied to clipboard
Yuwei Fang, Mahmoud Khademi, Chenguang Zhu, Ziyi Yang, Reid Pryzant, Yichong Xu, Yao Qian, Takuya Yoshioka, Lu Yuan, Michael Zeng, Xuedong Huang
| Challenge: | Existing frameworks for Integrative AI lack flexibility and composability to handle multimodal tasks. |
| Approach: | They propose a configurable framework for Integrative AI that orchestrates multiple pre-trained models to conduct complex multimodal tasks. |
| Outcome: | The proposed framework achieves impressive results on zero-shot multimodal tasks . it can communicate and personalize for users, and it can be used in a multimodal agent . |