Papers by Xin Yao

37 papers
Gradient-based Intra-attention Pruning on Pre-trained Language Models (2023.acl-long)

Copied to clipboard

Challenge: Pre-trained language models are computationally expensive and slow in inference due to their large sizes.
Approach: They propose a structured pruning method which combines pruning with knowledge distillation to yield highly effective models.
Outcome: The proposed method outperforms other pruning methods in sparsity regimes while maintaining 93% 99% performance.
The Elephant in the Room: Exploring the Role of Neutral Words in Language Model Group-Agnostic Debiasing (2025.findings-acl)

Copied to clipboard

Challenge: Large Language Models (LLMs) are increasingly integrated into our daily lives, raising ethical concerns, especially about perpetuating stereotypes.
Approach: They propose a method that incorporates a neutral word semantics-based loss function to alleviate the deterioration of the LMS during debiasing.
Outcome: The proposed method alleviates the deterioration of the Language Modeling Score (LMS) by incorporating a neutral word semantics-based loss function.
Compressing Large-Scale Transformer-Based Models: A Case Study on BERT (2021.tacl-1)

Copied to clipboard

Challenge: Popular pre-trained Transformers have improved performance for various NLP tasks by sizable margins, but are too resource-hungry and computation-intensive to suit low-capacity devices or applications with strict latency requirements.
Approach: They present a literature review of the compression of Transformers, focusing on the popular BERT model, which has attracted considerable research attention.
Outcome: The proposed models improve Sentiment analysis, paraphrase detection, machine reading comprehension, question answering, text summarization, and other tasks by sizable margins.
SGA-MCTS: Decoupling Planning from Execution via Training-Free Atomic Experience Retrieval (2026.findings-acl)

Copied to clipboard

Challenge: a new framework casts LLM planning as non-parametric retrieval, but high latency of inference-time search and supervised fine-tuning are limitations.
Approach: They propose a framework that casts LLM planning as non-parametric retrieval . they leverage Monte Carlo Tree Search to explore the solution space .
Outcome: Empirical results show that SGA-MCTS can match the performance of SOTA systems without task-specific fine-tuning.
GMH: A General Multi-hop Reasoning Model for KG Completion (2021.emnlp-main)

Copied to clipboard

Challenge: Knowledge graphs are incomplete with many facts missing, causing performance bottlenecks in many applications.
Approach: They propose a general multi-hop reasoning task that can be formulated as a search process and can be extended to long-distance reasoning scenarios.
Outcome: The proposed model improves on baselines in short and long distance reasoning scenarios.
ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching (2026.findings-acl)

Copied to clipboard

Challenge: Existing autoregressive models for dialogue generation suffer from high latency and stability issues.
Approach: They propose a non-autoregressive (NAR) zero-shot spoken dialogue generation model based on flow-matching.
Outcome: The proposed model outperforms existing models in speech generation due to poor speech intelligibility and turn-taking precision.
CODE-MVP: Learning to Represent Source Code from Multiple Views with Contrastive Pre-Training (2022.findings-naacl)

Copied to clipboard

Challenge: Recent studies have focused on code representation learning, which aims to represent the semantics of source code into distributed vectors.
Approach: They propose to integrate different views with the natural-language description of source code into a unified framework with Multi-View contrastive Pre-training.
Outcome: The proposed model outperforms state-of-the-art models on three downstream tasks over five datasets.
Program Transfer for Answering Complex Questions over Knowledge Bases (2022.acl-long)

Copied to clipboard

Challenge: Program induction for complex questions over knowledge bases relies on a large number of parallel question-program pairs for the given KB, but the gold program annotations are usually lacking, making learning difficult.
Approach: They propose an approach to leverage program annotations on rich KBs as external supervision signals to aid program induction for low-resourced KB.
Outcome: The proposed approach outperforms SOTA methods on ComplexWebQuestions and WebQuestionSP.
CKnowEdit: A New Chinese Knowledge Editing Dataset for Linguistics, Facts, and Logic Error Correction in LLMs (2025.acl-long)

Copied to clipboard

Challenge: CKnowEdit is the first-ever knowledge editing dataset designed to correct linguistic, factual, and logical errors in Large Language Models.
Approach: They propose a Chinese knowledge editing dataset to correct linguistic, factual, and logical errors in Large Language Models.
Outcome: The proposed dataset highlights the challenges that LLMs face in mastering Chinese . CKnowEdit can correct linguistic, factual, and logical errors in Chinese, the authors show .
LLMs Trust Humans More, That’s a Problem! Unveiling and Mitigating the Authority Bias in Retrieval-Augmented Generation (2025.acl-long)

Copied to clipboard

Challenge: Large language models (LLMs) generate outputs that stray from user input or contravene established knowledge.
Approach: They propose a new phenomenon, Authority Bias, where LLMs favor one knowledge source over the other . they propose atomic information that generates conflicts and a Conflict Detection Enhanced Query framework .
Outcome: The proposed framework reduces Authority bias in large language models . it detects conflicts, performs credibility assessment on conflicting paragraphs, and detects perturbed text .
Modeling Temporal-Modal Entity Graph for Procedural Multimodal Machine Comprehension (2022.acl-long)

Copied to clipboard

Challenge: Procedural Multimodal Documents organize textual instructions and corresponding images step by step.
Approach: They propose a novel temporal-modal entity Graph for comprehending PMDs . they propose encoding and reasoning modules to capture textual and visual entities .
Outcome: The proposed model can capture textual and visual entities and trace their temporal-modal evolution.
AgenticEval: Toward Agentic and Self-Evolving Safety Evaluation of Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Existing static safety evaluation methods are ill-equipped to address dynamic nature of AI risks and evolving regulations, creating a critical safety gap.
Approach: They propose a new paradigm of agentic safety evaluation reframing evaluation as a continuous and self-evolving process rather than a one-time audit.
Outcome: The proposed framework shows a consistent decline in model safety as the evaluation hardens.
PaCoRe: Learning to Scale Test-Time Compute with Parallel Coordinated Reasoning (2026.acl-long)

Copied to clipboard

Challenge: Parallel Coordinated Reasoning (PaCoRe) overcomes a central limitation of contemporary language models: their inability to scale test-time compute (TTC) far beyond sequential reasoning under a fixed context window.
Approach: They propose a training-and-inference framework to overcome a central limitation of language models: their inability to scale test-time compute (TTC) under a fixed context window.
Outcome: The proposed model scales to multi-million-token effective TTC without exceeding context limits.
Interpretable and Low-Resource Entity Matching via Decoupling Feature Learning from Decision Making (2021.acl-long)

Copied to clipboard

Challenge: Entity Matching (EM) aims at recognizing entity records that denote the same real-world object.
Approach: They propose a novel EM framework that consists of Heterogeneous Information Fusion and Key Attribute Tree Induction to decouple feature representation from matching decision.
Outcome: The proposed framework outperforms SOTA EM models on 6 public datasets and 3 industrial datasets.
AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning (2025.emnlp-demos)

Copied to clipboard

Challenge: Large language model agents have enabled GUI-based automation, but their deployment is limited by noisy data, poor generalization, and lack of support for non-English GUIs.
Approach: They propose an 8B-parameter GUI agent built for robust and efficient on-device GUI interaction.
Outcome: The proposed GUI agent achieves promising performance on five public benchmarks and proposed Chinese benchmark CAGUI.
A Survey of Post-Training Scaling in Large Language Models (2025.acl-long)

Copied to clipboard

Challenge: Large language models (LLMs) have demonstrated proficiency in understanding and generating human natural languages.
Approach: They propose a framework for scaling large language models using supervised fine-tuning, RLxF and test-time compute methodologies.
Outcome: The proposed model can be used to understand and generate human natural languages.
KoRC: Knowledge Oriented Reading Comprehension Benchmark for Deep Text Understanding (2023.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks for deep text understanding have encountered two major limitations . most require human annotation of knowledge, which leads to limited knowledge coverage .
Approach: They propose a benchmark to help readers understand a document with prior knowledge . they use massive knowledge bases to guide annotators and large language models to construct knowledgable questions .
Outcome: The proposed benchmarks have limited knowledge coverage and use choices or spans as answers, which results in narrow answer space.
Probabilistic Tree-of-thought Reasoning for Answering Knowledge-intensive Complex Questions (2023.findings-emnlp)

Copied to clipboard

Challenge: Large language models (LLMs) are capable of answering knowledge-intensive complex questions with chain-of-thought reasoning.
Approach: They propose a method to solve complex questions with a tree-of-thought approach using parametric knowledge and retrieved external knowledge to augment CoT reasoning.
Outcome: The proposed approach outperforms SOTA methods on three Complex QA datasets under the open-domain setting.
Can LLMs Hear the Dogwhistle? (2026.findings-acl)

Copied to clipboard

Challenge: Existing safety benchmarks focus on explicitly harmful content, but ignore context-dependent expressions such as dogwhistles.
Approach: They propose a benchmark for evaluating LLM safety under dogwhistle-driven prompts . their findings expose a blind spot in current safety evaluation practices .
Outcome: The proposed benchmark compared safety performance with toxic terms using dogwhistle-driven prompts.
SelfAug: Mitigating Catastrophic Forgetting in Retrieval-Augmented Generation via Distribution Self-Alignment (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing solutions for supervised fine-tuning often lead to catastrophic forgetting, where models lose their previously acquired knowledge and general capabilities.
Approach: They propose a self-distribution alignment method that aligns input sequence logits to preserve the model’s semantic distribution, thereby mitigating catastrophic forgetting and improving downstream performance.
Outcome: The proposed method achieves a superior balance between downstream learning and general capability retention.
Probing the Safety Robustness of LLMs in Latent Space (2026.acl-long)

Copied to clipboard

Challenge: Despite substantial progress in safety alignment techniques, aligned large language models can still produce unsafe responses under minor internal perturbations.
Approach: They introduce Activation Steering Attack (ASA) and leverage the Negative Log-Likelihood (NLL) as a diagnostic signal to probe the local sensitivity of safety behaviors in latent space.
Outcome: The proposed method is model-agnostic and supervision-free, enabling a general and reproducible diagnostic metric for analyzing safety robustness.
SimPBL: A Multi-Agent Framework for Project-Based Learning (2026.acl-long)

Copied to clipboard

Challenge: Existing LLMs provide partial assistance without modeling these roles, and overly comprehensive help can reduce learner autonomy.
Approach: They propose a multi-agent framework with an orchestrator agent that provides adaptive scaffolding from interaction logs and collaborator agents that support project work through boundary-aware collaboration.
Outcome: The proposed framework improves learner examination scores by 14% . it is based on a multi-agent framework with an orchestrator agent .
P²Net: Parallel Pointer-based Network for Key Information Extraction with Complex Layouts (2025.findings-acl)

Copied to clipboard

Challenge: Existing methods for key information extraction are based on a limited set of entity categories and fixed layouts.
Approach: They propose a large-scale, human-annotated dataset for key information extraction . it is based on a human-annotated layout and 1,162 entity categories . they propose 'parallel pointer-based network' that leverages implicit relationships .
Outcome: Experiments on widely-used datasets show that the proposed model outperforms state-of-the-art methods while maintaining fast inference speeds.
DeepPlanner: Scaling Planning Capability for Deep Research Agents via Advantage Shaping (2026.findings-acl)

Copied to clipboard

Challenge: Existing approaches to planning involve implicit planning or introduce explicit planners without systematically optimizing the planning stage.
Approach: They propose an end-to-end RL framework that enhances the planning capabilities of deep research agents.
Outcome: Experiments show that DeepPlanner improves planning quality and achieves state-of-the-art results under a lower training budget.
AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora (2026.acl-long)

Copied to clipboard

Challenge: Existing knowledge graph construction frameworks require predefined schemas, limiting their scalability and domain coverage.
Approach: They propose a framework for fully autonomous knowledge graph construction that eliminates the need for predefined schemas.
Outcome: The proposed framework outperforms state-of-the-art models on multi-hop QA tasks and enhances LLM factuality.
MEIT: Multimodal Electrocardiogram Instruction Tuning on Large Language Models for Report Generation (2025.findings-acl)

Copied to clipboard

Challenge: Recent studies have focused on classifying cardiac conditions using ECG data but have overlooked ECG report generation, which is time-consuming and requires clinical expertise.
Approach: They propose a Multimodal ECG Instruction Tuning framework that extends the capability of large language models (LLMs) for the task.
Outcome: The proposed framework outperforms open-source LLMs and LLM backbones across two large-scale ECG datasets.
Chart2Code53: A Large-Scale Diverse and Complex Dataset for Enhancing Chart-to-Code Generation (2025.emnlp-main)

Copied to clipboard

Challenge: Existing Chart2code-related training datasets suffer from limited scale, limited type coverage, and inadequate complexity.
Approach: They propose to synthesize chart2code-related training datasets using web plotting code and chart images to address these challenges.
Outcome: The proposed dataset exhibits the greatest diversity and higher complexity compared to other open-source Chart2code related datasets.
RecMem: Recurrence-based Memory Consolidation for Efficient and Effective Long-Running LLM Agents (2026.findings-acl)

Copied to clipboard

Challenge: Existing memory systems invoke LLMs to extract episodic and semantic memory, and this leads to substantial token consumption.
Approach: They propose a method that stores incoming interactions in a subconscious memory layer and encodes them using lightweight embedding models for retrieval.
Outcome: Experiments show that RecMem reduces the memory construction token cost of three SOTA memory systems by up to 87% while exceeding their accuracy.
Pre-training Distillation for Large Language Models: A Design Space Exploration (2025.acl-long)

Copied to clipboard

Challenge: Knowledge distillation (KD) aims to transfer knowledge from a large teacher model to a smaller student model for model compression.
Approach: They extend knowledge distillation to the pre-training phase of large language models . they first conduct an experiment using a teacher LLM to distill a 1.9B student LLM .
Outcome: The proposed model can be used to distill a 1.9B student model using a teacher LLM.
Compilable Neural Code Generation with Compiler Feedback (2022.findings-acl)

Copied to clipboard

Challenge: Existing deep-learning approaches model code generation as text generation, but few of them account for compilability of the generated programs.
Approach: They propose a three-stage pipeline utilizing compiler feedback for compilable code generation to improve compilability.
Outcome: The proposed pipeline improves compilability of generated programs by combining compiler feedback, language model fine-tuning, and compilable discrimination.
VisKoP: Visual Knowledge oriented Programming for Interactive Knowledge Base Question Answering (2023.acl-demo)

Copied to clipboard

Challenge: Existing knowledge base question answering systems that parse natural language questions into knowledge oriented program language (KoPL) .
Approach: They propose a knowledge base question answering system that integrates human into the loop to edit and debug queries.
Outcome: The proposed system can debug and edit knowledge base questions on a million-entity-level . it provides auto-completion for its knowledge base schema and user interaction can fix a large portion of wrong KoPL programs to acquire the correct answer.
OAgents: An Empirical Study of Building Effective Agents (2025.findings-emnlp)

Copied to clipboard

Challenge: a recent study shows that agent research practices are far from standard, rigorous . lack of a standard evaluation protocol makes previous works not reproducible, authors say .
Approach: They conduct an empirical study on the GAIA benchmark to investigate agent design choices . they find that lack of a standard evaluation protocol makes previous works not reproducible .
Outcome: The proposed framework achieves state-of-the-art performance among open-source projects.
Soul-Mix: Enhancing Multimodal Machine Translation with Manifold Mixup (2024.acl-long)

Copied to clipboard

Challenge: Multimodal machine translation (MMT) aims to improve the performance of machine translation with the help of visual information.
Approach: They propose a multimodal machine translation mixup method that integrates visual information into conventional text-only neural machine translation systems.
Outcome: The proposed method outperforms existing models on a multi-directional dataset with fewer parameters and achieves new state-of-the-art performance.
PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models (2024.acl-long)

Copied to clipboard

Challenge: generative large language models (LLMs) exhibit surprising capability and integrate previous tasks into a unified text generation formulation.
Approach: They propose a privacy evaluation benchmark to quantify the privacy leakage of language models.
Outcome: The proposed benchmark compares PPLMs with different privacy implementations to find out how privacy leakage is handled.
BatonVoice: An Operationalist Framework for Enhancing Controllable Speech Synthesis with Linguistic Intelligence from LLMs (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches often fail to leverage the linguistic intelligence of Large Language Models (LLMs) Existing models lack the ability to follow text instructions for controllable Text-to-Speech (TTS).
Approach: They propose a framework where an LLM acts as a conductor, understanding user instructions and generating a textual plan - explicit vocal features.
Outcome: The proposed model outperforms open- and closed-source models in speech synthesis and achieves zero-shot cross-lingual generalization.
TiMem: Temporal-Hierarchical Memory Consolidation for Long-Horizon Conversational Agents (2026.findings-acl)

Copied to clipboard

Challenge: Existing memory frameworks provide limited support for temporally structured information across hierarchical levels, leading to fragmented memories and unstable long-horizon personalization.
Approach: They propose a temporal–hierarchical memory framework that organizes conversations through a Temporal Memory Tree.
Outcome: The proposed framework outperforms baselines while reducing the recalled memory length by 52.20%.
Untangle the KNOT: Interweaving Conflicting Knowledge and Reasoning Skills in Large Language Models (2024.lrec-main)

Copied to clipboard

Challenge: Existing studies have explained to what extent LLMs extract conflicting knowledge from the provided text, but they neglect the necessity to reason with conflicting information.
Approach: They construct a dataset for knowledge conflict resolution examination in the form of question answering that divides reasoning with conflicting knowledge into three levels.
Outcome: The proposed dataset enables analysis of reasoning with conflicting knowledge in the form of question answering.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations