Papers by Mao Yang

51 papers
VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models (2024.emnlp-main)

Copied to clipboard

Challenge: Recent research has focused on pushing weight-only quantization to extremely low-bit due to numerical representation limitations.
Approach: They propose a vector-based quantization approach that pushes LLMs to extremely low-bit . they propose scalar-based weight quantization that reduces memory requirements and optimizes storage costs .
Outcome: The proposed method reduces model quantization perplexity by 0.01-0.34 on LLaMA-2, 0.38-0.68 on mistral-7B, 4.41-7.34, on llaMA-3 on QA tasks on average.
Cross-Lingual Cross-Target Stance Detection with Dual Knowledge Distillation Framework (2023.emnlp-main)

Copied to clipboard

Challenge: Existing studies on stance detection were conducted mainly in English due to the low-resource problem in most non-English languages.
Approach: They propose to use a cross-lingual teacher and a teacher to transfer knowledge from source to target language to bridge the discrepancy between languages.
Outcome: The proposed framework bridges the discrepancy between languages and generalizes the knowledge to unseen targets in target language.
LadaBERT: Lightweight Adaptation of BERT through Hybrid Model Compression (2020.coling-main)

Copied to clipboard

Challenge: Existing models that use knowledge distillation are memory-intensive and latency-prohibitive . Existing solutions that use this knowledge distilling framework are expensive .
Approach: They propose a solution that uses weight pruning, matrix factorization and knowledge distillation to learn a smaller model.
Outcome: The proposed model reduces the training overheads by an order of magnitude on public datasets while preserving state-of-the-art accuracy.
From Charts to Code: A Hierarchical Benchmark for Multimodal Models (2026.acl-long)

Copied to clipboard

Challenge: Chart2Code is a new benchmark for evaluating the natural language to chart code generation capabilities of large multimodal models.
Approach: They introduce Chart2Code, a new benchmark for evaluating the natural language to chart code generation capabilities of large multimodal models.
Outcome: The proposed benchmark is the first to scale task complexity while capturing diverse scenarios.
Towards a Unified Multi-Dimensional Evaluator for Text Generation (2022.emnlp-main)

Copied to clipboard

Challenge: Existing evaluation frameworks for natural language generation are dominated by similarity-based metrics.
Approach: They propose a multi-dimensional evaluator for natural language generation that integrates multiple dimensions into one evaluer.
Outcome: The proposed evaluator improves on three typical NLG tasks and improves with external knowledge.
SpeechNet: Weakly Supervised, End-to-End Speech Recognition at Industrial Scale (2022.emnlp-industry)

Copied to clipboard

Challenge: End-to-end automatic speech recognition systems require thousands of hours of manual annotation and heavyweight computation to perform inference.
Approach: They propose to use a third-party ASR system as a weak supervision source and labeling functions derived from implicit user feedback to reduce human labor.
Outcome: The proposed system improves word-error rate and speed up 600% over third-party ASR.
Data Mixing Agent: Learning to Re-weight Domains for Continual Pre-training (2026.acl-long)

Copied to clipboard

Challenge: Existing methods for reweighting data mixtures rely on manual designation with certain heuristics based on intuition or empirical results.
Approach: They propose a model-based framework that learns to re-weight domains by reinforcement learning on large quantities of data mixing trajectories with corresponding feedback from an evaluation environment.
Outcome: The proposed framework outperforms baselines in achieving balanced performance across source and target fields and domain spaces without retraining.
Non-parallel Accent Transfer based on Fine-grained Controllable Accent Modelling (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing accent transfer methods rely on parallel data or speech recognition models.
Approach: They propose to use mutual information learning to disentangle accent features and control the accent of the generated speech during the inference time.
Outcome: The proposed framework achieves superior performance to baseline models in accentedness and audio quality.
FastCuRL: Curriculum Reinforcement Learning with Stage-wise Context Scaling for Efficient Training R1-like Reasoning Models (2025.findings-emnlp)

Copied to clipboard

Challenge: Improving training efficiency remains a challenge in large-scale Reinforcement Learning (RL).
Approach: They propose a curriculum RL framework with stage-wise context scaling to improve RL training efficiency.
Outcome: The proposed framework outperforms state-of-the-art reasoning models on five benchmarks and achieves 49.6% accuracy on AIME 2024.
Scented-EAE: Stage-Customized Entity Type Embedding for Event Argument Extraction (2024.findings-acl)

Copied to clipboard

Challenge: Existing methods for incorporating entities into EAE rely on prompts or NER . weak semantic associations due to missing role-entity correspondence cues . one-sided semantic understanding relying solely on argument role semantics a problem .
Approach: They propose an EAE model with stage-customized entity type embedding to explore the role of entity types.
Outcome: The proposed model achieves state-of-the-art performance on mainstream benchmarks and robustness in low-resource settings.
DecBERT: Enhancing the Language Understanding of BERT with Causal Attention Masks (2022.findings-naacl)

Copied to clipboard

Challenge: Experimental results show that Transformer Encoder model can't automatically capture word order, so explicit position embeddings are required to be fed into the target model.
Approach: They propose a Transformer-based language model DecBERT that uses a causal attention mask to capture word order.
Outcome: The proposed model improves on the GLUE language understanding benchmark and accelerates the pre-training process.
Knowledge Context Modeling with Pre-trained Language Models for Contrastive Knowledge Graph Completion (2024.findings-acl)

Copied to clipboard

Challenge: Text-based knowledge graph completion methods neglect knowledge contexts in inferring process.
Approach: They propose a framework which models the knowledge context as additional prompts with pre-trained language models for knowledge graph completion.
Outcome: The proposed framework achieves state-of-the-art on FB15k-237, WN18RR and Wikidata5M datasets.
Fewer is More: Boosting Math Reasoning with Reinforced Context Pruning (2024.emnlp-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) have shown impressive capabilities, yet they struggle with math reasoning.
Approach: They propose a coarse-to-fine pruner that prunes unimportant tokens to fit the context window.
Outcome: The proposed approach outperforms prompting baselines across various LLMs and 5 math datasets and achieves 4.55% absolute improvements without any fine-tuning.
Skeletons Matter: Dynamic Data Augmentation for Text-to-Query (2025.emnlp-main)

Copied to clipboard

Challenge: Existing studies focus on a single query language, resulting in limited generalizability . a new task paradigm is proposed to unify semantic parsing tasks across different query languages .
Approach: They propose a task paradigm that unifies parsing tasks across query languages . they identify query skeletons as a shared optimization target of Text-to-Query tasks .
Outcome: The proposed method achieves state-of-the-art performance using only a small amount of synthesized data.
SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction (2022.naacl-main)

Copied to clipboard

Challenge: Existing methods for relation extraction only implicitly learn to model relevant contexts and entity types while being trained for RE.
Approach: They propose to explicitly teach the model to capture relevant contexts and entity types by supervising and augmenting intermediate steps (SAIS) for RE.
Outcome: The proposed method outperforms the runner-up method on three benchmarks by 5.04% . textual contexts and entity types are the major information sources that lead to the success of previous approaches.
ObfusLM: Privacy-preserving Language Model Service against Embedding Inversion Attacks (2025.acl-long)

Copied to clipboard

Challenge: Recent studies show that obfuscation techniques for MLaaS are susceptible to embedding inversion attacks (EIAs).
Approach: They propose a model obfuscation framework that protects client inputs from embedding inversion attacks by obliviously obbing models.
Outcome: The proposed framework outperforms existing works in utility by 10% with a nearly 80% resistance rate against embedding inversion attacks.
Can Large Language Models Effectively Support Decision-Making in Sudden Emergencies? (2026.findings-acl)

Copied to clipboard

Challenge: Existing research has focused on the earlier stages of emergency response . lack of suitable datasets for reliable and compliance-aware decision-oriented modeling and evaluation is limiting current research .
Approach: They propose a first real-world emergency decision-making dataset EDM-Bench . they propose 'rule-enhanced reasoning framework' that integrates external regulatory knowledge with constrained inference mechanisms to improve both decision safety and interpretability.
Outcome: The proposed framework improves decision safety and interpretability by integrating regulatory knowledge with constrained inference mechanisms.
Improving Relation Extraction with Knowledge-attention (D19-1)

Copied to clipboard

Challenge: Existing attention mechanisms are data-driven, but most are data driven.
Approach: They propose a knowledge-attention encoder which integrates prior knowledge from external lexical resources into deep neural networks for relation extraction task.
Outcome: The proposed system outperforms existing CNN, RNN, and self-attention based models on a large-scale relation extraction dataset.
Learning to Rewrite: Generalized LLM-Generated Text Detection (2025.acl-long)

Copied to clipboard

Challenge: Existing detectors for Large Language Models (LLMs) struggle to generalize in open-world settings.
Approach: They propose a framework to detect LLM-generated text with exceptional generalization to unseen domains by reinforcing LLMs’ inherent rewriting tendencies.
Outcome: The proposed framework outperforms state-of-the-art detection methods by 23.04% in AUROC, 35.10% for out-of distribution tests, and 48.66% under adversarial attacks.
Refining Corpora from a Model Calibration Perspective for Chinese Spelling Correction (2024.findings-acl)

Copied to clipboard

Challenge: Chinese Spelling Correction (CSC) lacks large-scale high-quality corpora due to labor-intensive labeling of spelling errors in real-life writing or typing scenarios.
Approach: They propose to use OCR/ASR-based generation to refine Chinese Spelling Correction models on random replacement-based corpora and filter them based on prediction confidence.
Outcome: The proposed model outperforms existing models on three widely-used benchmarks while significantly alleviating over-correction.
NetSafe: Exploring the Topological Safety of Multi-agent System (2025.findings-acl)

Copied to clipboard

Challenge: Large language models (LLMs) have fueled significant progress in intelligent Multi-agent Systems (MAS), with expanding academic and industrial applications.
Approach: They propose a framework that unifies diverse MAS workflows via iterative RelCom interactions to enable generalized analysis.
Outcome: The proposed framework unifies diverse MAS workflows via iterative RelCom interactions to enable generalized analysis.
Comprehensive Study: How the Context Information of Different Granularity Affects Dialogue State Tracking? (2021.acl-long)

Copied to clipboard

Challenge: Dialogue state tracking (DST) plays a key role in task-oriented dialogue systems to monitor the user’s goal.
Approach: They propose to use scratch-based and previous-based strategies to track dialogue state . they explore how different granularities affect dialogue state tracking .
Outcome: The scratch-based strategy obtains each slot value by inquiring all the dialogue history, while the previous-based method is not very useful for long-dependency dialogue state tracking.
Diversity Helps Jailbreak Large Language Models (2025.naacl-long)

Copied to clipboard

Challenge: Existing methods for jailbreaking large language models rely on laborious human engineering and whitebox access to model internals.
Approach: They propose a method that instructs large language models to deviate from prior context and generate harmful outputs by instructing them to deviat from previous attacks.
Outcome: The proposed method achieves a 62.83% higher success rate in compromising ten leading chatbots, while using only 12.9% of the queries.
Multi-stage Distillation Framework for Cross-Lingual Semantic Similarity Matching (2022.findings-naacl)

Copied to clipboard

Challenge: Existing studies have shown that cross-lingual knowledge distillation can improve the performance of pre-trained models for cross-linguistic similarity matching tasks.
Approach: They propose a multi-stage distillation framework for constructing a small-size but high-performance cross-lingual model using contrastive learning, bottleneck, and parameter recurrent strategies.
Outcome: The proposed model can compress the size of XLM-R and MiniLM by more than 50% while the performance is only reduced by about 1%.
HyperCRS: Hypergraph-Aware Multi-Grained Preference Learning to Burst Filter Bubbles in Conversational Recommendation System (2025.findings-acl)

Copied to clipboard

Challenge: Existing methods to analyze filter bubbles in the static recommendation environment are unable to burst them during user interactions.
Approach: They propose a paradigm to learn multi-grained user preferences during dynamic user-system interactions via natural language conversations to burst filter bubbles.
Outcome: The proposed paradigm achieves state-of-the-art performance and the superior of bursting filter bubbles in the conversational recommendation system.
Knowledge-Augmented Question Error Correction for Chinese Question Answer System with QuestionRAG (2025.emnlp-industry)

Copied to clipboard

Challenge: Large language models struggle with input errors, often failing to interpret user intent or altering the original question’s structure (over-correction).
Approach: They propose a framework that uses reinforcement learning to address misinterpretation and over-correction by integrating external knowledge with the input.
Outcome: The proposed framework unlocks the full potential of LLMs for the question correction task.
Context or Knowledge is Not Always Necessary: A Contrastive Learning Framework for Emotion Recognition in Conversations (2023.findings-acl)

Copied to clipboard

Challenge: Existing studies focus on modeling context-sensitive dependencies and knowledge-sensitive dependences.
Approach: They propose a framework based on contrastive learning called CKCL to distinguish utterances for better vector representations.
Outcome: The proposed framework outperforms state-of-the-art models on four datasets.
Adversarial Preference Learning for Robust LLM Alignment (2025.findings-acl)

Copied to clipboard

Challenge: Modern language models rely on Reinforcement Learning from Human Feedback (RLHF) to encourage safe behaviors, but they remain vulnerable to adversarial attacks due to three key limitations: (1) the inefficiency and high cost of human annotation; (2) the vast diversity of potential adversarials; and (3) the risk of feedback bias and reward hacking.
Approach: They propose an iterative adversarial training method that incorporates three key innovations to address these challenges.
Outcome: Experiments on Mistral-7B-Instruct-v0.3 show that the proposed method significantly enhances robustness and reduces harmful outputs from 5.88% to 0.43%.
QAEval: Mixture of Evaluators for Question-Answering Task Evaluation (2025.acl-long)

Copied to clipboard

Challenge: Existing QA evaluation methods struggle with open-ended and unstructured responses.
Approach: They propose a hybrid framework that combines rule-based reliability with LLM-based adaptability to overcome these challenges.
Outcome: The proposed framework outperforms existing models like GPT-4o and Claude-3 in accuracy and cost.
Rethinking Task-Oriented Dialogue Systems: From Complex Modularity to Zero-Shot Autonomous Agent (2024.acl-long)

Copied to clipboard

Challenge: Task-oriented dialogue systems are designed to be composed of several functional modules, but lacks a general-purpose instruction-following language model.
Approach: They propose a fully zero-shot autonomous TOD agent that leverages a general-purpose instruction-following language model to decide what to do at each dialogue turn.
Outcome: The proposed agent can perform tasks in real-life scenarios with a general-purpose instruction-following language model.
Causal-Debias: Unifying Debiasing in Pretrained Language Models and Fine-tuning via Causal Invariant Learning (2023.acl-long)

Copied to clipboard

Challenge: Existing methods to remove unwanted stereotypical associations from pretrained language models (PLMs) are often focused on removing unwanted stereotypes from PLMs.
Approach: They propose a framework to remove unwanted stereotypical associations in pretrained language models . they propose bias-relevant factors are causal, while labelrelevant factors causal .
Outcome: The proposed framework reduces stereotypical associations after PLMs are fine-tuned . the proposed framework mitigates bias from a causal invariant perspective .
Parameter-efficient Continual Learning Framework in Industrial Real-time Text Classification System (2022.naacl-industry)

Copied to clipboard

Challenge: Existing continual learning methods use data replay, parameter isolation and regularization to mitigate catastrophic forgetting.
Approach: They propose a parameter-efficient continual learning framework that updates parameters offline and then trains using an online regularization method.
Outcome: The proposed framework reduces catastrophic forgetting and saves the model with the changed parameters instead of all parameters.
Enhancing Multilingual Reasoning via Steerable Model Merging (2026.findings-acl)

Copied to clipboard

Challenge: Model merging is an effective technique for composing the capabilities of a multilingual model and a reasoning model.
Approach: They propose a model merging framework that modulates the contribution of each source model.
Outcome: Experiments show that the proposed model merging framework outperforms strong baselines on multilingual reasoning benchmarks across 21 different languages.
Rationales Are Not Silver Bullets: Measuring the Impact of Rationales on Model Performance and Reliability (2025.findings-acl)

Copied to clipboard

Challenge: Existing studies have shown that training language models with rationales augmentation is beneficial, but this view does not hold consistently.
Approach: They conduct comprehensive investigations to thoroughly inspect the impact of rationales on model performance and a novel perspective of model reliability.
Outcome: The proposed method outperforms untrained models in several areas and provides informative regulations on the broad utilization of rationales.
A Knowledge Regularized Hierarchical Approach for Emotion Cause Analysis (D19-1)

Copied to clipboard

Challenge: Emotion cause analysis aims to identify the reasons behind emotions . previous models focus on learning architecture with local textual information .
Approach: They propose a method to extract emotion cause with hierarchical neural model and knowledge-based regularizations by sentiment lexicon and common knowledge.
Outcome: The proposed method outperforms baselines on two public datasets in different languages and outperformed competitive baselines by 2.08%.
RAFT: Realistic Attacks to Fool Text Detectors (2024.emnlp-main)

Copied to clipboard

Challenge: Large language models (LLMs) have exhibited remarkable fluency across tasks, but their unethical applications are unclear.
Approach: They propose a grammar error-free black-box attack that exploits LLM embeddings at the word-level while preserving original text quality.
Outcome: The proposed attack compromises all detectors across domains and is transferable across source models.
Gold-Medal-Level Olympiad Geometry Solving with Efficient Heuristic Auxiliary Constructions (2026.acl-long)

Copied to clipboard

Challenge: Existing methods for geometry theorem proving in Euclidean geometry are challenging and require a neural network to perform.
Approach: They propose a method for adding auxiliary points in geometry that runs on CPUs without relying on neural network-based inference.
Outcome: The proposed method achieves silver-medal-level human performance on IMO-30 benchmark.
Efficient Data Labeling by Hierarchical Crowdsourcing with Large Language Models (2025.coling-main)

Copied to clipboard

Challenge: Large language models (LLMs) have been gaining attention for their impressive performance in in-context dialogues.
Approach: They propose a hierarchical framework that leverages multiple LLMs for efficient data labeling under budget constraints.
Outcome: The proposed framework outperforms human labelers and GPT-4 in terms of accuracy and efficiency.
Improving Chinese Spelling Check by Character Pronunciation Prediction: The Effects of Adaptivity and Granularity (2022.emnlp-main)

Copied to clipboard

Challenge: Chinese spelling check (CSC) is a fundamental NLP task that detects and corrects spelling errors in Chinese texts.
Approach: They propose an auxiliary task of Chinese pronunciation prediction to improve CSC . they propose adaptive weighting schemes and a delicate correction strategy .
Outcome: The proposed auxiliary task improves Chinese pronunciation prediction on three benchmarks.
Target-Oriented Relation Alignment for Cross-Lingual Stance Detection (2023.findings-acl)

Copied to clipboard

Challenge: Existing work on cross-lingual stance detection has ignored the inconsistency in the occurrences and distributions of targets between languages, which consequently degrades the performance of stance detector in low-resource languages.
Approach: They propose a fine-grained method which considers both target-level associations and language-level alignments to learn the in-language and cross-language associations.
Outcome: The proposed method is compared with competing methods under variant settings and shows that it performs better in low-resource languages.
MiMoTable: A Multi-scale Spreadsheet Benchmark with Meta Operations for Table Reasoning (2025.coling-main)

Copied to clipboard

Challenge: Existing benchmarks for table reasoning are incomplete due to the complexity of the tables and user questions in real-world applications.
Approach: They propose a Multi-scale spreadsheet benchmark with Meta operations for Table reasoning that incorporates two key features and a new criterion with six categories of meta operations for measuring the difficulty of each question.
Outcome: The proposed model outperforms Claude-3.5-Sonnet with 77.4% accuracy on the existing benchmarks.
Finding the Pillars of Strength for Multi-Head Attention (2023.acl-long)

Copied to clipboard

Challenge: Recent studies have revealed some issues of Multi-Head Attention (MHA) e.g., redundancy and over-parameterization.
Approach: They propose to train attention heads with a self-supervised group constraint to focus on an essential but distinctive feature subset.
Outcome: The proposed method achieves significant performance gains on three well-established tasks while significantly compressing parameters.
Unsupervised Multi-Granularity Summarization (2022.findings-emnlp)

Copied to clipboard

Challenge: Experimental results confirm the substantial superiority of GranuSum on multi-granularity summarization over strong baselines.
Approach: They propose to rank events by their salience and annotate a benchmark for GranuSum that contains multiple summaries at different granularities for each document cluster.
Outcome: The proposed framework is capable of producing multi-granular summaries in unsupervised manner over strong baselines.
Seq2Path: Generating Sentiment Tuples as Paths of a Tree (2022.findings-acl)

Copied to clipboard

Challenge: Existing generative methods for extracting sentiment tuples do not have orders between the t-uples . a novel parallel generative framework for ABSA is proposed .
Approach: They propose a parallel generative framework to generate sentiment tuples as paths of a tree . they train the model with an independent target and introduce a discriminative token .
Outcome: The proposed method achieves state-of-the-art on AOPE, ASTE, TASD, UABSA, ACOS . it trains with the loss of ordinary Seq2Seq averaged over paths, and inferences automatically select valid paths.
Why Can Distillation Work with Limited Resources? A Systematic Study (2026.findings-acl)

Copied to clipboard

Challenge: Recent advances in large language models have driven reasoning performance . low-resource distillation can boost models' performance, but a framework is missing .
Approach: They conduct a controlled experiment to find out why low-resource distillation can boost model performance . they find that distillation enhances the presence of advanced cognitive behaviors .
Outcome: The proposed model shows more flexible reasoning, the authors show . they show that distillation enhances the presence of advanced cognitive behaviors .
SSR-Zero: Simple Self-Rewarding Reinforcement Learning for Machine Translation (2026.findings-acl)

Copied to clipboard

Challenge: Large language models (LLMs) have demonstrated remarkable capabilities in machine translation, but most MT-specific LLMs rely heavily on external supervision during training.
Approach: They propose a reinforcement learning framework for machine translation that is reference-free and relies solely on self-judging rewards.
Outcome: The proposed framework outperforms existing LLMs and larger general LLM models on English Chinese translation benchmarks and performs competitively with leading closed-source systems.
Voice Query Auto Completion (2021.emnlp-main)

Copied to clipboard

Challenge: Existing methods fail to complete voice queries from incomplete prefixes because they use orthographic prefix and substrings instead of the true phonetic prefix.
Approach: They propose to condition QAC approaches on intermediate transcriptions to complete voice queries.
Outcome: The proposed method obtains an 18% relative improvement over previous methods on a speech-enabled smart television with real-life voice search traffic.
From Experience to Skill: Multi-Agent Generative Engine Optimization via Reusable Strategy Learning (2026.findings-acl)

Copied to clipboard

Challenge: Generative engines (GEs) are replacing ranked links with citation-grounded answers . current methods are unable to accumulate or transfer effective strategies across tasks and engines .
Approach: They propose a multi-agent framework where planning, editing, and fidelity-aware evaluation serve as the execution layer.
Outcome: The proposed framework outperforms heuristic baselines in visibility and citation fidelity on three mainstream engines.
DisCo: Distilled Student Models Co-training for Semi-supervised Text Mining (2023.emnlp-main)

Copied to clipboard

Challenge: Existing text mining models are fine-tuned by fine-timing a large pre-trained language model (PLM) in downstream tasks.
Approach: They propose a semi-supervised learning framework for fine-tuning a cohort of small student models generated from a large pre-trained language model using knowledge distillation.
Outcome: The proposed framework outperforms baseline models on semi-supervised text classification and extractive summarization tasks while maintaining comparable performance.
InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training (2021.naacl-main)

Copied to clipboard

Challenge: Existing methods for learning cross-lingual representations are lacking in the field of NLP.
Approach: They propose a framework that formulates cross-lingual language model pre-training as maximizing mutual information between multilingual-multi-granularity texts.
Outcome: The proposed approach improves cross-lingual transferability on benchmarks.
KuiLeiXi: a Chinese Open-Ended Text Adventure Game (2021.acl-demo)

Copied to clipboard

Challenge: Recent advances in pre-trained language models have made it possible to generate human-like text.
Approach: They propose to integrate an open-ended text adventure game in Chinese, named KuiLeiXi, where players interact with the AI until the plot goals are reached.
Outcome: The proposed game lacks incentives and relies on players to explore on their own.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations