Papers by Yao Song

V-ALPHASOCIAL: Benchmark and Self-Reflective Chain-of-Thought Generation for Visual Social Commonsense Reasoning (2025.findings-acl)

Challenge:	Existing models suffer from temporal redundancy when leveraged under dynamic settings.
Approach:	They propose a temporal knowledge graph extrapolation method which solves temporal redundancy issues by using cyclic rules to capture more information lurking in TKGs.
Outcome:	The proposed model captures more information lurking in TKGs, and also mines and properly leverages acyclic rules, which has not been explored by existing models.

Copied to clipboard

Zongyu Lin, Zhikun Xu, Xiaohan Song, Yixin Wan, Xingcheng Yao, Tsung-Han Lin, Selina Song, Pranav Subbaraman, Ben Zhou, Kai-Wei Chang, Yizhou Sun

Reasoning

Challenge:	Social commonsense reasoning is a multimodal task that requires both textual and visual cues.
Approach:	They propose a method that integrates visual cues into social commonsense reasoning tasks.
Outcome:	The proposed method improves social commonsense reasoning on a multimodal foundation model.

Generate-on-Graph: Treat LLM as both Agent and KG for Incomplete Knowledge Graph Question Answering (2024.emnlp-main)

Copied to clipboard

Yao Xu, Shizhu He, Jiabei Chen, Zihao Wang, Yangqiu Song, Hanghang Tong, Guang Liu, Jun Zhao, Kang Liu

Structured Data in NLP

Challenge:	Existing methods to integrate LLMs with Knowledge Graphs (KGs) however, these methods are often incomplete to cover all the knowledge required to answer questions.
Approach:	They propose to integrate LLMs with Knowledge Graphs (KGs) to address insufficient knowledge and hallucination issues in Large Language Models.
Outcome:	The proposed method outperforms existing methods on two datasets.

ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching (2026.findings-acl)

Copied to clipboard

Han Zhu, Wei Kang, Liyong Guo, Zengwei Yao, Fangjun Kuang, Weiji Zhuang, Zhaoqing Li, Zhifeng Han, Dong Zhang, Xin Zhang, Xingchen Song, Lingxuan Ye, Long Lin, Daniel Povey

Towards the Law of Capacity Gap in Distilling Language Models (2025.acl-long)

Challenge:	Existing autoregressive models for dialogue generation suffer from high latency and stability issues.
Approach:	They propose a non-autoregressive (NAR) zero-shot spoken dialogue generation model based on flow-matching.
Outcome:	The proposed model outperforms existing models in speech generation due to poor speech intelligibility and turn-taking precision.

Copied to clipboard

Chen Zhang, Qiuchi Li, Dawei Song, Zheyu Ye, Yan Gao, Yao Hu

Leveraging Dual Process Theory in Language Agent Framework for Real-time Simultaneous Human-AI Collaboration (2025.acl-long)

Challenge:	Language model (LM) distillation aims at distilling knowledge in a large teacher LM to a small student one.
Approach:	They propose to use the law of capacity gap to distill knowledge from a large teacher to a small student model.
Outcome:	The proposed model outperforms other language models on a larger scale by using the law of capacity gap inducted from a preliminary study on small-scale (3B) LMs.

Copied to clipboard

Shao Zhang, Xihuai Wang, Wenhao Zhang, Chaoran Li, Junru Song, Tingyu Li, Lin Qiu, Xuezhi Cao, Xunliang Cai, Wen Yao, Weinan Zhang, Xinbing Wang, Ying Wen

Linguistic Theories

Challenge:	Large language models (LLMs) excel in turn-by-turn human-AI collaboration but struggle with simultaneous tasks requiring real-time interaction.
Approach:	They propose a language agent framework that integrates System 1 and System 2 for efficient real-time simultaneous human-AI collaboration.
Outcome:	The proposed framework improves on existing LLM-based agents and human collaborators by integrating Theory of Mind and asynchronous reflection to infer human intentions and perform reasoning-based autonomous decisions.

How do Words Contribute to Sentence Semantics? Revisiting Sentence Embeddings with a Perturbation Method (2023.eacl-main)

Copied to clipboard

Wenlin Yao, Lifeng Jin, Hongming Zhang, Xiaoman Pan, Kaiqiang Song, Dian Yu, Dong Yu, Jianshu Chen

Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstruction (2026.findings-acl)

Challenge:	Existing studies on sentence representation learning focus on human annotation, but they neglect the critical property that essential contents should contribute to sentence semantics more than non-essential contents when encoding a sentence.
Approach:	They propose a perturbation method for unsupervised semantic analysis that uses a sentence compression metric to adapt sentence compression datasets for automatic evaluation.
Outcome:	The proposed method can capture the main semantics of sentences better than several SOTA unsupervised sentence embedding models.

Copied to clipboard

Xu Shen, Qi Zhang, Song Wang, Zhen Tan, Xinyu Zhao, Laura Yao, Vaishnav Tadiparthi, Hossein Nourkhiz Mahjoub, Ehsan Moradi Pari, Kwonjoon Lee, Tianlong Chen

From Scenes to Elements: Multi-Granularity Evidence Retrieval for Verifiable Multimodal RAG (2026.findings-acl)

Challenge:	Large Language Model based multi-agent systems (MAS) excel at collaborative problem solving but remain brittle to cascading errors.
Approach:	They propose a metacognitive framework that enables step-level error detection and self-correction in Large Language Model based multi-agent systems (MAS) .
Outcome:	The proposed framework outperforms baselines on the Who When benchmark and delivers consistent gains on AgentErrorBench.

Copied to clipboard

Guanhua Chen, Chuyue Huang, Yutong Yao, Shudong Liu, Xueqing Song, Lidia S. Chao, Derek F. Wong

Improving Meta-learning for Low-resource Text Classification and Generation via Memory Imitation (2022.acl-long)

Challenge:	Existing multimodal Retrieval-Augmented Generation (RAG) systems retrieve evidence at coarse granularities, making failures unverifiable.
Approach:	They propose a multimodal benchmark that features real-world landmarks with annotations across multiple viewpoints and a framework that treats visual elements as first-class retrieval units through three stages: element-level detection and classification, multi-granularity cross-modal alignment for evidence retrieval, and attribution-constrained generation.
Outcome:	The proposed framework achieves up to 29.2% improvement over six strong baselines for this task.

Copied to clipboard

Yingxiu Zhao, Zhiliang Tian, Huaxiu Yao, Yinhe Zheng, Dongkyu Lee, Yiping Song, Jian Sun, Nevin Zhang

Information Extraction & Text Mining

Challenge:	Building models of natural language processing (NLP) is challenging in low-resource scenarios where limited data are available.
Approach:	They propose a memory imitation meta-learning method that enhances the model’s reliance on support sets for task adaptation.
Outcome:	The proposed method outperforms baselines on both text classification and generation tasks.

NarraSum: A Large-Scale Dataset for Abstractive Narrative Summarization (2022.findings-emnlp)

Copied to clipboard

Chao Zhao, Faeze Brahman, Kaiqiang Song, Wenlin Yao, Dian Yu, Snigdha Chaturvedi

Text Generation

Challenge:	Existing studies focus on summarizing news documents or structured documents.
Approach:	They propose to use a large-scale narrative summarization dataset to encourage research . they find there is a performance gap between humans and the models on NarraSum .
Outcome:	The proposed dataset shows that humans and state-of-the-art models perform poorly when summarizing a narrative . it contains 122K narratives collected from synopses of movies and TV episodes with diverse genres .

Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal (2024.acl-long)

Copied to clipboard

Jianheng Huang, Leyang Cui, Ante Wang, Chengyi Yang, Xinting Liao, Linfeng Song, Junfeng Yao, Jinsong Su

Representation Learning with Ordered Relation Paths for Knowledge Graph Completion (D19-1)

Challenge:	Existing methods to train LLMs on previous training data are not feasible in real-world applications because of catastrophic forgetting.
Approach:	They propose a framework that uses the LLM to generate synthetic instances for rehearsal and refine the instance outputs based on the synthetic inputs.
Outcome:	The proposed framework achieves superior or comparable performance compared to conventional rehearsal-based approaches while being more data-efficient.

Copied to clipboard

Yao Zhu, Hongzhi Liu, Zhonghai Wu, Yang Song, Tao Zhang

Enhancing Abstractiveness of Summarization Models through Calibrated Distillation (2023.findings-emnlp)

Challenge:	Existing knowledge graphs are incomplete and lack the order of relations in paths.
Approach:	They propose a method which takes relation paths into account but ignores order of relations in paths which is important for reasoning.
Outcome:	The proposed method performs better than state-of-the-art methods on two benchmark datasets.

Copied to clipboard

Hwanjun Song, Igor Shalyminov, Hang Su, Siffi Singh, Kaisheng Yao, Saab Mansour

Summarization

Challenge:	Existing methods to generate abstractive summarizations are slow and abstractive, but we propose a novel approach to enhance the level of abstractiveness without sacrificing the informativeness of generated summaries.
Approach:	They propose a novel approach to enhance the level of abstractiveness without sacrificing the informativeness of generated summaries by exposing diverse pseudo summary with two supervision to the student model.
Outcome:	The proposed method outperforms previous methods in abstractive summarization distillation, producing highly abstractive and informative summaries.

Empathy in Diversity: Personalized Depression and Anxiety Therapy via Dialogue State Tracking and Patient-Aware Planning (2026.acl-long)

Copied to clipboard

Xinwei Yang, Junyi Fan, Yuqing Liu, Jiaxuan Wang, Jiashuai Zhang, Hongru Liang, Wenqiang Lei, Yao Song

ParetoRAG: Leveraging Sentence-Context Attention for Robust and Efficient Retrieval-Augmented Generation (2025.findings-emnlp)

Challenge:	Recent efforts have turned to large language models (LLMs) as therapeutic agents for psychological therapy tasks, yet robustness across diverse patients remains underexplored.
Approach:	They propose a realistic role-play protocol for evaluating therapeutic dialogue agents and a de-identified, expert-annotated corpus of therapist–patient dialogues.
Outcome:	The proposed framework outperforms baselines on therapeutic outcomes and dialogue quality while improving conversational efficiency.

Copied to clipboard

Ruobing Yao, Yifei Zhang, Shuang Song, Yuhan Liu, Neng Gao, Chenyang Tu

Responsible & Trustworthy NLP

Challenge:	Retrieval-augmented generation systems face persistent challenges in retrieval inefficiency and the inability of LLMs to filter out irrelevant information.
Approach:	They propose an unsupervised framework that optimizes RAG systems through sentence-level refinement guided by the Pareto principle.
Outcome:	The proposed framework achieves dual improvements in retrieval precision and generation quality without additional training or API resources while using only 40% of the tokens compared to traditional approaches.

Model-Based Imaginative Planning for Embodied Agents (2026.acl-long)

Copied to clipboard

Junru Song, Hengzhe Jin, Yucong Huang, Tingsong Jiang, Weien Zhou, Feifei Wang, Yang Yang, Ying Wen, Wen Yao

DeepPlanner: Scaling Planning Capability for Deep Research Agents via Advantage Shaping (2026.findings-acl)

Challenge:	a lightweight world model converts raw pixels into object-centric symbolic states amenable to language-based reasoning . IMPLEMENT is a framework for grounding language agents in visual embodied environments .
Approach:	They propose a model-based reasoning framework that enables frozen large language models to perform imaginative planning.
Outcome:	The proposed framework can be used to ground language agents in visual embodied environments.

Copied to clipboard

Wei Fan, Wenlin Yao, Zheng Li, Feng Yao, Xin Liu, Liang Qiu, Qingyu Yin, Yangqiu Song, Bing Yin

Reasoning

Challenge:	Existing approaches to planning involve implicit planning or introduce explicit planners without systematically optimizing the planning stage.
Approach:	They propose an end-to-end RL framework that enhances the planning capabilities of deep research agents.
Outcome:	Experiments show that DeepPlanner improves planning quality and achieves state-of-the-art results under a lower training budget.

AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora (2026.acl-long)

Copied to clipboard

Jiaxin Bai, Wei Fan, Qi Hu, Qing Zong, Chunyang Li, Hong Ting Tsang, Hongyu Luo, Yauwai Yim, Haoyu Huang, Xiao Zhou, Feng Qin, Tianshi Zheng, Xi Peng, Xin Yao, Huiwen Yang, Leijie Wu, JI Yi, Gong Zhang, Renhai Chen, Yangqiu Song

MoDification: Mixture of Depths Made Easy (2025.naacl-long)

Challenge:	Existing knowledge graph construction frameworks require predefined schemas, limiting their scalability and domain coverage.
Approach:	They propose a framework for fully autonomous knowledge graph construction that eliminates the need for predefined schemas.
Outcome:	The proposed framework outperforms state-of-the-art models on multi-hop QA tasks and enhances LLM factuality.

Copied to clipboard

Chen Zhang, Meizhi Zhong, Qimeng Wang, Xuantao Lu, Zheyu Ye, Chengqiang Lu, Yan Gao, Yao Hu, Kehai Chen, Min Zhang, Dawei Song

Green & Sustainable NLP

Challenge:	Long-context efficiency is a trending topic in large language model (LLM) serving.
Approach:	They propose a method to combine long-context efficiency and mixture of depths to bring down both latency and memory.
Outcome:	The proposed method achieves 1.2 speedup in latency and 1.8 reduction in memory compared to original LLMs especially in long-context applications.

InFoBench: Evaluating Instruction Following Ability in Large Language Models (2024.findings-acl)

Copied to clipboard

Yiwei Qin, Kaiqiang Song, Yebowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang, Xuansheng Wu, Fei Liu, Pengfei Liu, Dong Yu

Learning-by-Narrating: Narrative Pre-Training for Zero-Shot Dialogue Comprehension (2022.acl-short)

Challenge:	Existing methods for evaluating Large Language Models (LLMs) ability to follow instructions have not been able to provide a detailed analysis of their compliance with instructions.
Approach:	They propose a new metric for evaluating Large Language Models' ability to follow instructions and a benchmark for DRFR.
Outcome:	The proposed metric and benchmark compared with traditional scoring methods and explores annotation sources including human experts, crowd-sourced workers, and GPT-4.

Copied to clipboard

Chao Zhao, Wenlin Yao, Dian Yu, Kaiqiang Song, Dong Yu, Jianshu Chen

HAHE: Hierarchical Attention for Hyper-Relational Knowledge Graphs in Global and Local Level (2023.acl-long)

Challenge:	Existing models for dialogue comprehension are not available for the pre-training of such a model.
Approach:	They propose a narrative-guided pre-training strategy that learns by narrating key information from a dialogue input.
Outcome:	The proposed model performs better on four dialogue-based tasks and is comparable to existing models.

Copied to clipboard

Haoran Luo, Haihong E, Yuhao Yang, Yikai Guo, Mingzhi Sun, Tianyu Yao, Zichen Tang, Kaiyang Wan, Meina Song, Wei Lin

Bridging Continuous and Discrete Spaces: Interpretable Sentence Representation Learning via Compositional Operations (2023.emnlp-main)

Challenge:	Existing research on HKGs rarely models the graphical and sequential structure of HKG, limiting their representation.
Approach:	They propose a Hierarchical Attention model for HKG Embedding that includes global-level and local-level attention to model the graphical structure of HKGs.
Outcome:	The proposed model achieves state-of-the-art performance on HKG standard datasets and addresses the issue of HKG multi-position prediction for the first time.

Copied to clipboard

James Y. Huang, Wenlin Yao, Kaiqiang Song, Hongming Zhang, Muhao Chen, Dong Yu

GAST: Gradient-aligned Sparse Tuning of Large Language Models with Data-layer Selection (2026.eacl-long)

Challenge:	Existing approaches to learn sentence embeddings do not capture the semantic similarity of sentences.
Approach:	They propose a framework that integrates compositional sentence operations into the embedding space and optimizes operator networks and a bottleneck encoder-decoder model to produce meaningful and interpretable sentence embeddables.
Outcome:	The proposed framework improves the interpretability of sentence embeddings on four textual generation tasks while maintaining strong performance on traditional semantic similarity tasks.

Copied to clipboard

Kai Yao, Zhenghan Song, Kaixin Wu, Mingjie Zhong, Danzhao Cheng, Zhaorui Tan, Yixin Ji, Penglei Gao

Rethinking the Video Sampling and Reasoning Strategies for Temporal Sentence Grounding (2022.findings-emnlp)

Challenge:	Existing methods focus on layer-selective and data-selectory fine-tuning, but ignore the fact that different data points contribute varying degrees to distinct model layers.
Approach:	They propose a method that performs selective fine-tuning at both data and layer dimensions as integral components of a unified optimization strategy.
Outcome:	Experiments show that the proposed method outperforms baseline methods in terms of performance and performance.

Copied to clipboard

Jiahao Zhu, Daizong Liu, Pan Zhou, Xing Di, Yu Cheng, Song Yang, Wenzheng Xu, Zichuan Xu, Yao Wan, Lichao Sun, Zeyu Xiong

Salience Allocation as Guidance for Abstractive Summarization (2022.emnlp-main)

Challenge:	Existing methods for temporal sentence grounding ignore two crucial issues . 1) Boundary-bias: the video downsampling process may lose these two frames . 2) Reasoning-biases: such incorrect new boundary frames lead to the reasoning bias .
Approach:	They propose a siamese sampling mechanism to generate additional contextual frames . they use a reasoning strategy to learn the inter-relationship among these frames a .
Outcome:	Extensive experiments demonstrate the effectiveness of a new siamese sampling network on three challenging datasets.

Copied to clipboard

Fei Wang, Kaiqiang Song, Hongming Zhang, Lifeng Jin, Sangwoo Cho, Wenlin Yao, Xiaoyang Wang, Muhao Chen, Dong Yu

Text Generation

Challenge:	Abstractive summarization models implicitly learn to capture the salient information from scratch.
Approach:	They propose a method that uses salience expectation to guide abstractive summarization by averaging salient content to a fixed threshold.
Outcome:	The proposed method can be easily adapted to documents with various abstractiveness and achieves high performance.

Creative and Context-Aware Translation of East Asian Idioms with GPT-4 (2024.findings-emnlp)

Copied to clipboard

Kenan Tang, Peiyang Song, Yao Qin, Xifeng Yan

Multilinguality

Challenge:	figurative language is a challenge for human translators, who often choose a context-aware translation . a set of commonly used idioms condenses its figurativ meaning into a few characters .
Approach:	They evaluate whether GPT-4 can generate high-quality translations using Pareto-optimal prompting strategies that outperform translation engines from Google and DeepL.
Outcome:	The proposed translations outperform translation engines from Google and DeepL at low cost.

MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning (2024.naacl-long)

Copied to clipboard

Fuxiao Liu, Xiaoyang Wang, Wenlin Yao, Jianshu Chen, Kaiqiang Song, Sangwoo Cho, Yaser Yacoob, Dong Yu

Fine-grained Conversational Decoding via Isotropic and Proximal Search (2023.emnlp-main)

Challenge:	Existing large language models have limited ability to perform tasks effectively.
Approach:	They propose a large-scale multimodal chart instruction dataset with 600k instances supporting diverse tasks and chart types.
Outcome:	The proposed LMM achieves state-of-the-art performance on existing chart QA benchmarks.

Copied to clipboard

Yuxuan Yao, Han Wu, Qiling Xu, Linqi Song

Hierarchical Heterogeneous Graph Representation Learning for Short Text Classification (2021.emnlp-main)

Challenge:	Existing text decoding methods are not tailoring for dialogue generation.
Approach:	They propose a fine-grained conversational decoding method that generates a semantic-concentrated response while maintaining informativeness and discrimination against the context.
Outcome:	The proposed method outperforms existing decoding strategies in the dialogue field across both automatic and human evaluation metrics.

Copied to clipboard

Yaqing Wang, Song Wang, Quanming Yao, Dejing Dou

Structured Data in NLP

Challenge:	Short text classification is a fundamental task in natural language processing.
Approach:	They propose a new method called SHINE which is based on graph neural network for short text classification.
Outcome:	The proposed method outperforms state-of-the-art methods on benchmark short text datasets.

PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models (2024.acl-long)

Copied to clipboard

Haoran Li, Dadi Guo, Donghao Li, Wei Fan, Qi Hu, Xin Liu, Chunkit Chan, Duanyi Yao, Yuan Yao, Yangqiu Song

Responsible & Trustworthy NLP

Challenge:	generative large language models (LLMs) exhibit surprising capability and integrate previous tasks into a unified text generation formulation.
Approach:	They propose a privacy evaluation benchmark to quantify the privacy leakage of language models.
Outcome:	The proposed benchmark compares PPLMs with different privacy implementations to find out how privacy leakage is handled.

DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation (2022.acl-long)

Copied to clipboard

Wei Chen, Yeyun Gong, Song Wang, Bolun Yao, Weizhen Qi, Zhongyu Wei, Xiaowu Hu, Bartuer Zhou, Yi Mao, Weizhu Chen, Biao Cheng, Nan Duan

Is Parameter Collision Hindering Continual Learning in LLMs? (2025.coling-main)

Challenge:	Existing pre-trained dialog models shed light on various downstream tasks in natural language processing (NLP).
Approach:	They propose a dialog pre-training framework that introduces latent variables into the enhanced encoder-decoder pre-train framework to increase relevance and diversity of responses.
Outcome:	The proposed model achieves state-of-the-art on personaChat, DailyDialog, and DSTC7-AVSD datasets.

Copied to clipboard

Shuo Yang, Kun-Peng Ning, Yu-Yang Liu, Jia-Yu Yao, Yong-Hong Tian, Yi-Bing Song, Li Yuan