Papers by Ming Gao

43 papers
SpanProto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognition (2022.emnlp-main)

Copied to clipboard

Challenge: Existing methods for few-shot Named Entity Recognition ignore entity boundaries and are time-consuming . a seminal span-based prototypical network solves the problem using two stages: span extraction and mention classification.
Approach: They propose a seminal span-based prototypical network that tackles few-shot NER . they transform sequential tags into a global boundary matrix and use prototypical learning .
Outcome: The proposed model outperforms strong baselines over multiple benchmarks.
Semantics of the Unwritten: The Effect of End of Paragraph and Sequence Tokens on Text Generation with GPT2 (2021.acl-srw)

Copied to clipboard

Challenge: Experimental results show that pre-trained language model GPT2 can generate better continuations by learning to generate the in the fine-tuning stage.
Approach: They conduct experiments on an English essay dataset using Chinese-GPT2 . they find that the model can generate better continuations by learning to generate the in the fine-tuning stage.
Outcome: The pre-trained language model GPT2 can generate better continuations by learning to generate the in the fine-tuning stage.
Evaluating and Enhancing the Robustness of Code Pre-trained Models through Structure-Aware Adversarial Samples Generation (2023.findings-emnlp)

Copied to clipboard

Challenge: Pre-trained code models have made significant strides in the field of neural code intelligence, but they are susceptible to adversarial attacks that subtly modify the input sequence and can impair generalization.
Approach: They propose a set of novel robustness evaluation methods based on the intrinsic structure of the code to explore the impact of imperceptible perturbation.
Outcome: The proposed methods have demonstrated their effectiveness across a wide range of models and tasks, and are able to predict the performance of perturbed models.
Make Prompt-based Black-Box Tuning Colorful: Boosting Model Generalization from Three Orthogonal Perspectives (2024.lrec-main)

Copied to clipboard

Challenge: Large language models (LLMs) have shown increasing power on NLP tasks. however, tuning these models for downstream tasks usually requires exorbitant costs.
Approach: They propose a black-box tuning technique that optimizes task-specific prompts without accessing gradients and hidden representations.
Outcome: The proposed method improves performance under few-shot learning scenarios.
SocialBench: Sociality Evaluation of Role-Playing Conversational Agents (2024.findings-acl)

Copied to clipboard

Challenge: Existing studies on role-playing agents have focused on enhancing their conversational capability, role-specific knowledge and style, but there has been a gap in assessing their social intelligence.
Approach: They propose a benchmark to evaluate the sociality of role-playing agents using LLMs.
Outcome: The proposed benchmark is constructed from various sources and covers a wide range of 500 characters and over 6,000 question prompts and 30,800 multi-turn role-playing utterances.
Can Pretrained Language Models Derive Correct Semantics from Corrupt Subwords under Noise? (2023.starsem-1)

Copied to clipboard

Challenge: Existing studies have shown that Pretrained Language Models (PLMs) perform poorly under noise due to subword segmentation.
Approach: They propose a framework for subword segmentation that provides a systematic categorization of segmentation corruption under noise and evaluation protocols by generating contrastive datasets with canonical-noisy word pairs.
Outcome: The proposed framework provides a systematic categorization of segmentation corruption under noise and evaluation protocols by generating contrastive datasets with canonical-noisy word pairs.
CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challenge (2025.findings-acl)

Copied to clipboard

Challenge: Large language models (LLMs) have demonstrated remarkable capabilities, but their capabilities in cryptographic decryption tasks remain underexplored.
Approach: They propose a benchmark to evaluate the reasoning capabilities of large language models in cryptographic decryption tasks.
Outcome: The proposed benchmark examines the reasoning capabilities of large language models in cryptographic decryption tasks.
Aligning Large Language Models with Implicit Preferences from User-Generated Content (2025.acl-long)

Copied to clipboard

Challenge: Existing preference learning methods rely heavily on curated data from humans or advanced LLMs, which is costly and difficult to scale.
Approach: They propose a framework that leverages implicit preferences in unlabeled user-generated content to generate preference data.
Outcome: The proposed framework transforms user-generated content into user queries and generates responses from the policy model.
Knowledgeable In-Context Tuning: Exploring and Exploiting Factual Knowledge for In-Context Learning (2024.findings-naacl)

Copied to clipboard

Challenge: Existing studies have explored multiple aspects that affect the performance of large language models (LLMs) such as input-output mapping, extensive data resources, and the ability to train on labeled examples.
Approach: They propose a framework that injects knowledge into LLMs during continual self-supervised pre-training and judiciously selects examples with high knowledge relevance.
Outcome: The proposed framework outperforms baseline models and improves by more than 13% and 7% on text classification and question-answering tasks.
ProQA: Structural Prompt-based Pre-training for Unified Question Answering (2022.naacl-main)

Copied to clipboard

Challenge: Existing QA research on question answering is focused on specific question types, knowledge domains, or reasoning skills.
Approach: They propose a unified QA paradigm that solves various tasks through a single model.
Outcome: The proposed model improves QA-centric ability on 11 QA benchmarks.
Towards Unified Prompt Tuning for Few-shot Text Classification (2022.findings-emnlp)

Copied to clipboard

Challenge: Prompt-based fine-tuning has boosted performance of Pre-trained Language Models (PLMs) on few-shot text classification, but PLMs are unfamiliar with prompt-style expressions during pre-training, which limits the few- shot learning performance on downstream tasks.
Approach: They propose a framework for prompt-based fine-tuning that captures prompting semantics from non-target NLP datasets and propose 'Prompt-Options-Verbalizer' for joint prompt learning across different NLP tasks.
Outcome: Experiments show that the proposed framework outperforms state-of-the-art prompt-based fine-tuning frameworks on few-shot text classification tasks.
Towards Robust Few-Shot Relation Classification: Incorporating Relation Description with Agreement (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing approaches to recognize relational relationships with a few support samples are limited for unlimited queries.
Approach: They propose a simple but effective framework that uses relation descriptions as external knowledge to enhance the model’s comprehension of the relation semantics.
Outcome: The proposed framework outperforms strong baselines while being robust against various NOTA rates.
Meta-Learning Adversarial Domain Adaptation Network for Few-Shot Text Classification (2021.findings-acl)

Copied to clipboard

Challenge: Existing approaches for few-shot text classification rely on exploitation of lexical features and distributional signatures on training data, while neglecting to strengthen the model's ability to adapt to new tasks.
Approach: They propose a meta-learning framework integrated with an adversarial domain adaptation network to improve the model's adaptive ability and generate high-quality text embedding for new classes.
Outcome: The proposed framework outperforms the state-of-the-art models on four datasets and shows clear superiority over existing models.
DialCoT Meets PPO: Decomposing and Exploring Reasoning Paths in Smaller Language Models (2023.emnlp-main)

Copied to clipboard

Challenge: Chain-of-Thought prompting has improved the reasoning capabilities of Large Language Models (LLMs) but it is ineffective or detrimental to the performance on reasoning tasks in Smaller Language Model (SLMs) with less than 10 billion parameters.
Approach: They propose a Dialogue-guided Chain-of-Thought method to improve the reasoning capabilities of Large Language Models (LLMs) by generating intermediate reasoning steps in a dialogue format to guide the model to the final answer.
Outcome: The proposed method can achieve significant performance gains over state-of-the-art competitors on four arithmetic reasoning datasets.
A Neural Network Architecture for Program Understanding Inspired by Human Behaviors (2022.acl-long)

Copied to clipboard

Challenge: Existing studies for understanding programs do not take human behaviors as reference.
Approach: They propose a graph neural network model that takes human behaviors as reference in understanding programs.
Outcome: The proposed model performs better on code summarization and code clone detection tasks.
Simple but Effective Compound Geometric Operations for Temporal Knowledge Graph Completion (2024.acl-long)

Copied to clipboard

Challenge: Current methods embed factual knowledge into continuous vector space and apply geometric operations to learn potential patterns in temporal knowledge graphs.
Approach: They propose a temporal knowledge graph completion method that uses two geometric operations to learn missing facts in temporal graphs.
Outcome: The proposed method significantly outperforms existing temporal knowledge graph embedding models.
Intelligent Document Parsing: Towards End-to-end Document Parsing via Decoupled Content Parsing and Layout Grounding (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing methods fragment document parsing into pipeline of separated subtasks, resulting in incomplete semantics and error propagation.
Approach: They propose an end-to-end document parsing framework that leverages vision-language priors of MLLMs.
Outcome: The proposed method surpasses existing methods significantly in document parsing . it leverages the vision-language priors of MLLMs to decouple parse and layout grounding based on visual information.
TRUST: Towards Robust Social Bot Detection via Uncertainty-Guided Pseudo-Labeling and Graph Structure Purification (2026.findings-acl)

Copied to clipboard

Challenge: Existing graph-based detection models are vulnerable to deceptive message propagation, where bots deliberately interact with legitimate users.
Approach: They propose a framework to mitigate deceptive message propagation by node-level uncertainty estimation and graph structure purification.
Outcome: The proposed framework improves on three benchmark datasets and six GNN backbones on real-world social bots.
PA-RAG: RAG Alignment via Multi-Perspective Preference Optimization (2025.naacl-long)

Copied to clipboard

Challenge: Existing approaches to optimize RAG generators fail to align with RAG requirements thoroughly.
Approach: They propose a method for optimizing the RAG generator from multiple preference perspectives to align with RAG requirements comprehensively.
Outcome: The proposed method improves the performance of RAG generators by incorporating retrieved documents into the prompt.
TransPrompt: Towards an Automatic Transferable Prompting Framework for Few-shot Text Classification (2021.emnlp-main)

Copied to clipboard

Challenge: Recent studies show that prompts improve performance of large pre-trained language models for few-shot text classification.
Approach: They propose a prompt-based framework for few-shot learning that captures cross-task transferable knowledge and uses two de-biasing techniques to make it more task-agnostic and unbiased .
Outcome: The proposed framework outperforms strong baselines over multiple NLP tasks and datasets.
TransCoder: Towards Unified Transferable Code Representation Learning Inspired by Human Skills (2024.lrec-main)

Copied to clipboard

Challenge: Existing methods to fine-tune code intelligence models to individual tasks are costly and require large data sets.
Approach: They propose a Transferable fine-tuning strategy for Code representation learning that uses a tunable prefix encoder to capture cross-task and cross-language transferable knowledge and apply it to downstream adaptation.
Outcome: The proposed method can lead to superior performance on code-related tasks and encourage mutual reinforcement.
ARTIST: A Transformer-based Chinese Text-to-Image Synthesizer Digesting Linguistic and World Knowledge (2022.findings-emnlp)

Copied to clipboard

Challenge: Text-to-Image Synthesis (TIS) is a popular task to convert natural language texts into realistic images.
Approach: They propose a transformer-based Chinese text-to-image synthesizer for high-resolution image generation that incorporates linguistic and relational knowledge facts into the model to ensure better performance without the usage of ultra-large models.
Outcome: The proposed model outperforms existing models in Chinese with linguistic and relational knowledge facts.
CoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreference (2024.emnlp-main)

Copied to clipboard

Challenge: Existing studies have not noticed the safety risks of large language models . authors evaluated 1,400 questions in multi-turn dialogue coreference .
Approach: They are the first to evaluate LLM safety in multi-turn dialogue coreference . they created a dataset of 1,400 questions and tested five open-source models .
Outcome: The study shows that model safety decreases in multi-turn dialogue coreference scenarios . the highest success rate was with the LLaMA2-Chat-7b model, while the lowest was with mistral-7B-Instruct model .
Boosting Language Models Reasoning with Chain-of-Knowledge Prompting (2024.acl-long)

Copied to clipboard

Challenge: Recent studies have shown that Chain-of-Thought (CoT) prompting can be effective on complex reasoning tasks but generates unfaithful and unfactual reasoning chains.
Approach: They propose a chain-of-knowledge prompting that elicits Large Language Models to generate explicit pieces of knowledge evidence in the form of structure triple.
Outcome: The proposed method improves commonsense, factual, symbolic, and arithmetic reasoning tasks by estimating the reliability of the reasoning chains in terms of factuality and faithfulness.
Uncertainty-aware Parameter-Efficient Self-training for Semi-supervised Language Understanding (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for pre-trained language models rely on noisy data, which can be expensive if all parameters are updated.
Approach: They propose a self-training framework that incorporates Monte Carlo dropouts into the model and judiciously selects reliable pseudo-labeled examples based on confidence and certainty.
Outcome: The proposed framework improves performance and efficiency over multiple tasks over multiple datasets.
Knowledge Prompting in Pre-trained Language Model for Natural Language Understanding (2022.emnlp-main)

Copied to clipboard

Challenge: Existing knowledge-enhanced pre-trained language models (PLMs) introduce redundant factual knowledge from knowledge bases and require complex modules.
Approach: They propose a knowledge prompting-based PLM framework that incorporates factual knowledge into PLMs.
Outcome: The proposed framework can be flexibly combined with existing mainstream PLMs.
Large Language Models Are Poor Clinical Decision-Makers: A Comprehensive Benchmark (2024.emnlp-main)

Copied to clipboard

Challenge: Existing studies focus on evaluating large language models in close-ended QA tasks, but many clinical decisions involve answering open-ended questions without pre-set options.
Approach: They construct a benchmark to better understand large language models in the clinic . they use existing datasets to evaluate LLMs in clinical situations .
Outcome: The proposed model outperforms human experts in multiple medical tasks.
Context-Aware Conversation Thread Detection in Multi-Party Chat (D19-1)

Copied to clipboard

Challenge: In multi-party chat, it is common for multiple conversations to occur concurrently . a new model that automatically disentangles conversation threads is proposed .
Approach: They propose a Context-Aware Thread Detection model that automatically disentangles conversation threads in chat logs.
Outcome: The proposed model outperforms state-of-the-art models on four real-world chat logs.
MathFusion: Enhancing Mathematical Problem-solving of LLM through Instruction Fusion (2025.acl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) have shown impressive progress in mathematical problem-solving . current approaches to enhance mathematical reasoning focus on instance-level modifications .
Approach: They propose a framework that enhances mathematical reasoning through cross-problem instruction synthesis.
Outcome: The proposed framework boosts mathematical reasoning by 18.0 points while maintaining high data efficiency.
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention (2025.acl-long)

Copied to clipboard

Challenge: Long-context modeling is crucial for next-generation language models, but high computational cost of standard attention mechanisms poses significant computational challenges.
Approach: They propose a natively trained Sparse Attention mechanism that integrates algorithms with hardware-aligned optimizations to achieve efficient long-context modeling.
Outcome: The proposed model maintains or exceeds Full Attention models across general benchmarks, long-context tasks, and instruction-based reasoning.
KECP: Knowledge Enhanced Contrastive Prompting for Few-shot Extractive Question Answering (2022.emnlp-main)

Copied to clipboard

Challenge: Extractive Question Answering (EQA) is one of the most essential tasks in Machine Reading Comprehension (MRC).
Approach: They propose a framework that transforms extractive question answering into a non-autoregressive Masked Language Modeling (MLM) generation problem.
Outcome: The proposed framework outperforms state-of-the-art approaches in few-shot learning scenarios by a large margin.
FashionKLIP: Enhancing E-Commerce Image-Text Retrieval with Fashion Multi-Modal Conceptual Knowledge Graph (2023.acl-industry)

Copied to clipboard

Challenge: Recent advances in visual-language pre-trained (VLP) models have greatly improved cross-modal retrieval performance . however, the fine-grained interactions between objects from different modalities are far from well-established . e-commerce domain lacks sufficient training data and fine-granular cross-modulal knowledge .
Approach: They propose a visual-language pre-trained (VLP) image-text retrieval model that integrates cross-modal knowledge into the model to improve performance.
Outcome: The proposed model improves performance on e-commerce image-text retrieval task by a large margin.
TIGEr: Text-to-Image Grounding for Image Caption Evaluation (D19-1)

Copied to clipboard

Challenge: Existing metrics based on text-level comparisons fail to assess the quality of captions produced by machines.
Approach: They propose to use a machine-learned text-image grounding model to measure the accuracy of machine-generated captions and their correlation with human judgments.
Outcome: The proposed metric has higher consistency with human judgments and is more accurate than existing metrics.
Structure-aware Fine-tuning for Code Pre-trained Models (2024.lrec-main)

Copied to clipboard

Challenge: Existing CodePTMs are mainly structure-free and structurebased, but how to fine-tune them remains a challenge.
Approach: They propose a plug-and-play fine-tuning method that incorporates structural knowledge into pre-trained code models.
Outcome: The proposed method can benefit CodePTMs more with limited training data.
Is Cognition Consistent with Perception? Assessing and Mitigating Multimodal Knowledge Conflicts in Document Understanding (2025.emnlp-main)

Copied to clipboard

Challenge: Multimodal large language models (MLLMs) have shown impressive capabilities in document understanding due to different types of annotation noise in training.
Approach: They propose a method to reduce C&P knowledge conflicts across all tested MLLMs . they propose to use annotation noise to train models to understand document content .
Outcome: The proposed method reduces C&P knowledge conflicts across all tested MLLMs and enhances their performance in both cognitive and perceptual tasks.
InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment (2024.findings-acl)

Copied to clipboard

Challenge: Existing large language models (LLMs) can solve graph reasoning and generation tasks with parameter updates without sacrificing performance.
Approach: They propose a structured format verbalizer to unify all graph data into a universal code-like format, which can simply represent the graph without any external graph-specific encoders.
Outcome: The proposed framework outperforms GPT-4 and LLaMA2 in graph reasoning and generation tasks by more than 13% and 38%, respectively.
Pass-Tuning: Towards Structure-Aware Parameter-Efficient Tuning for Code Representation Learning (2023.findings-emnlp)

Copied to clipboard

Challenge: Code pre-trained models have been proposed and widely applied in the domain of code intelligence.
Approach: They propose a method that uses a plug-and-play graph neural network module as a tunable prefix to exploit structural information of source code.
Outcome: The proposed method exploits structural information of source code and could replace full fine-tuning.
MIND: A Large-scale Dataset for News Recommendation (2020.acl-main)

Copied to clipboard

Challenge: Personalized news recommendation is an important technique for personalized news service.
Approach: They propose to build a large-scale news recommendation dataset from Microsoft News . they demonstrate that news recommendation relies on the quality of news content understanding .
Outcome: The proposed dataset contains 1 million users and more than 160k English news articles, each of which has rich textual content such as title, abstract and body.
CAT-probing: A Metric-based Approach to Interpret How Pre-trained Models for Programming Language Attend Code Structure (2022.findings-emnlp)

Copied to clipboard

Challenge: Existing code pre-trained models fail to consider inherent characteristics of codes . Existing methods to interpret code pretrained model fail to take into account inherent characteristics .
Approach: They propose a probing method to quantitatively interpret how CodePTMs attend code structure.
Outcome: The proposed method denoises input code sequences and measures commonality between token-level attention scores and pair-wise distances between corresponding AST nodes.
REO-Relevance, Extraness, Omission: A Fine-grained Evaluation for Image Captioning (D19-1)

Copied to clipboard

Challenge: Existing metrics for image captioning evaluation provide an overall quality score, which is difficult to infer specific description errors.
Approach: They propose a fine-grained evaluation method REO for automatically measuring the performance of image captioning systems.
Outcome: The proposed method achieves higher consistency with human judgments and provides more intuitive evaluation results than other metrics.
Conjoin after Decompose: Improving Few-Shot Performance of Named Entity Recognition (2024.lrec-main)

Copied to clipboard

Challenge: Existing prompt-based NER models fail to detect entity boundaries, causing performance degradation.
Approach: They propose a model which consists of a BART encoder and a parabiotic decoder and propose ' boundary expansion strategy' to enhance the model's capability in entity type classification.
Outcome: The proposed model can achieve significant performance gains over state-of-the-art models.
When Gradient Descent Meets Derivative-Free Optimization: A Match Made in Black-Box Scenario (2023.findings-acl)

Copied to clipboard

Challenge: Large pre-trained language models (PLMs) are expensive and may not be open-sourced due to commercial considerations and potential risks of misuse.
Approach: They propose to introduce gradient descent into black-box tuning scenario . they propose a method which integrates gradient descent and derivative-free optimization .
Outcome: The proposed method achieves significant performance gains over previous state-of-the-art methods.
ECoK: Emotional Commonsense Knowledge Graph for Mining Emotional Gold (2024.findings-acl)

Copied to clipboard

Challenge: Existing knowledge graphs focus on the representation and reasoning of general factual knowledge, while there are significant deficiencies in the understanding and reasoning for emotional knowledge.
Approach: They propose a commonsense knowledge graph that can be used to represent emotional knowledge by combining theories from psychology, cognitive science, and linguistics.
Outcome: The proposed model surpasses GPT-4-Turbo in the emotion-related tasks.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations