Papers by Ming Gao
Copied to clipboard
| Challenge: | Existing methods for few-shot Named Entity Recognition ignore entity boundaries and are time-consuming . a seminal span-based prototypical network solves the problem using two stages: span extraction and mention classification. |
| Approach: | They propose a seminal span-based prototypical network that tackles few-shot NER . they transform sequential tags into a global boundary matrix and use prototypical learning . |
| Outcome: | The proposed model outperforms strong baselines over multiple benchmarks. |
Copied to clipboard
| Challenge: | Experimental results show that pre-trained language model GPT2 can generate better continuations by learning to generate the in the fine-tuning stage. |
| Approach: | They conduct experiments on an English essay dataset using Chinese-GPT2 . they find that the model can generate better continuations by learning to generate the in the fine-tuning stage. |
| Outcome: | The pre-trained language model GPT2 can generate better continuations by learning to generate the in the fine-tuning stage. |
Copied to clipboard
| Challenge: | Pre-trained code models have made significant strides in the field of neural code intelligence, but they are susceptible to adversarial attacks that subtly modify the input sequence and can impair generalization. |
| Approach: | They propose a set of novel robustness evaluation methods based on the intrinsic structure of the code to explore the impact of imperceptible perturbation. |
| Outcome: | The proposed methods have demonstrated their effectiveness across a wide range of models and tasks, and are able to predict the performance of perturbed models. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown increasing power on NLP tasks. however, tuning these models for downstream tasks usually requires exorbitant costs. |
| Approach: | They propose a black-box tuning technique that optimizes task-specific prompts without accessing gradients and hidden representations. |
| Outcome: | The proposed method improves performance under few-shot learning scenarios. |
Copied to clipboard
| Challenge: | Existing studies on role-playing agents have focused on enhancing their conversational capability, role-specific knowledge and style, but there has been a gap in assessing their social intelligence. |
| Approach: | They propose a benchmark to evaluate the sociality of role-playing agents using LLMs. |
| Outcome: | The proposed benchmark is constructed from various sources and covers a wide range of 500 characters and over 6,000 question prompts and 30,800 multi-turn role-playing utterances. |
Copied to clipboard
| Challenge: | Existing studies have shown that Pretrained Language Models (PLMs) perform poorly under noise due to subword segmentation. |
| Approach: | They propose a framework for subword segmentation that provides a systematic categorization of segmentation corruption under noise and evaluation protocols by generating contrastive datasets with canonical-noisy word pairs. |
| Outcome: | The proposed framework provides a systematic categorization of segmentation corruption under noise and evaluation protocols by generating contrastive datasets with canonical-noisy word pairs. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable capabilities, but their capabilities in cryptographic decryption tasks remain underexplored. |
| Approach: | They propose a benchmark to evaluate the reasoning capabilities of large language models in cryptographic decryption tasks. |
| Outcome: | The proposed benchmark examines the reasoning capabilities of large language models in cryptographic decryption tasks. |
Copied to clipboard
| Challenge: | Existing preference learning methods rely heavily on curated data from humans or advanced LLMs, which is costly and difficult to scale. |
| Approach: | They propose a framework that leverages implicit preferences in unlabeled user-generated content to generate preference data. |
| Outcome: | The proposed framework transforms user-generated content into user queries and generates responses from the policy model. |
Copied to clipboard
| Challenge: | Existing studies have explored multiple aspects that affect the performance of large language models (LLMs) such as input-output mapping, extensive data resources, and the ability to train on labeled examples. |
| Approach: | They propose a framework that injects knowledge into LLMs during continual self-supervised pre-training and judiciously selects examples with high knowledge relevance. |
| Outcome: | The proposed framework outperforms baseline models and improves by more than 13% and 7% on text classification and question-answering tasks. |
Copied to clipboard
| Challenge: | Existing QA research on question answering is focused on specific question types, knowledge domains, or reasoning skills. |
| Approach: | They propose a unified QA paradigm that solves various tasks through a single model. |
| Outcome: | The proposed model improves QA-centric ability on 11 QA benchmarks. |
Copied to clipboard
| Challenge: | Prompt-based fine-tuning has boosted performance of Pre-trained Language Models (PLMs) on few-shot text classification, but PLMs are unfamiliar with prompt-style expressions during pre-training, which limits the few- shot learning performance on downstream tasks. |
| Approach: | They propose a framework for prompt-based fine-tuning that captures prompting semantics from non-target NLP datasets and propose 'Prompt-Options-Verbalizer' for joint prompt learning across different NLP tasks. |
| Outcome: | Experiments show that the proposed framework outperforms state-of-the-art prompt-based fine-tuning frameworks on few-shot text classification tasks. |
Copied to clipboard
| Challenge: | Existing approaches to recognize relational relationships with a few support samples are limited for unlimited queries. |
| Approach: | They propose a simple but effective framework that uses relation descriptions as external knowledge to enhance the model’s comprehension of the relation semantics. |
| Outcome: | The proposed framework outperforms strong baselines while being robust against various NOTA rates. |
Copied to clipboard
| Challenge: | Existing approaches for few-shot text classification rely on exploitation of lexical features and distributional signatures on training data, while neglecting to strengthen the model's ability to adapt to new tasks. |
| Approach: | They propose a meta-learning framework integrated with an adversarial domain adaptation network to improve the model's adaptive ability and generate high-quality text embedding for new classes. |
| Outcome: | The proposed framework outperforms the state-of-the-art models on four datasets and shows clear superiority over existing models. |
Copied to clipboard
| Challenge: | Chain-of-Thought prompting has improved the reasoning capabilities of Large Language Models (LLMs) but it is ineffective or detrimental to the performance on reasoning tasks in Smaller Language Model (SLMs) with less than 10 billion parameters. |
| Approach: | They propose a Dialogue-guided Chain-of-Thought method to improve the reasoning capabilities of Large Language Models (LLMs) by generating intermediate reasoning steps in a dialogue format to guide the model to the final answer. |
| Outcome: | The proposed method can achieve significant performance gains over state-of-the-art competitors on four arithmetic reasoning datasets. |
Copied to clipboard
| Challenge: | Existing studies for understanding programs do not take human behaviors as reference. |
| Approach: | They propose a graph neural network model that takes human behaviors as reference in understanding programs. |
| Outcome: | The proposed model performs better on code summarization and code clone detection tasks. |
Copied to clipboard
| Challenge: | Current methods embed factual knowledge into continuous vector space and apply geometric operations to learn potential patterns in temporal knowledge graphs. |
| Approach: | They propose a temporal knowledge graph completion method that uses two geometric operations to learn missing facts in temporal graphs. |
| Outcome: | The proposed method significantly outperforms existing temporal knowledge graph embedding models. |
Copied to clipboard
| Challenge: | Existing methods fragment document parsing into pipeline of separated subtasks, resulting in incomplete semantics and error propagation. |
| Approach: | They propose an end-to-end document parsing framework that leverages vision-language priors of MLLMs. |
| Outcome: | The proposed method surpasses existing methods significantly in document parsing . it leverages the vision-language priors of MLLMs to decouple parse and layout grounding based on visual information. |
Copied to clipboard
| Challenge: | Existing graph-based detection models are vulnerable to deceptive message propagation, where bots deliberately interact with legitimate users. |
| Approach: | They propose a framework to mitigate deceptive message propagation by node-level uncertainty estimation and graph structure purification. |
| Outcome: | The proposed framework improves on three benchmark datasets and six GNN backbones on real-world social bots. |
Copied to clipboard
| Challenge: | Existing approaches to optimize RAG generators fail to align with RAG requirements thoroughly. |
| Approach: | They propose a method for optimizing the RAG generator from multiple preference perspectives to align with RAG requirements comprehensively. |
| Outcome: | The proposed method improves the performance of RAG generators by incorporating retrieved documents into the prompt. |
Copied to clipboard
| Challenge: | Recent studies show that prompts improve performance of large pre-trained language models for few-shot text classification. |
| Approach: | They propose a prompt-based framework for few-shot learning that captures cross-task transferable knowledge and uses two de-biasing techniques to make it more task-agnostic and unbiased . |
| Outcome: | The proposed framework outperforms strong baselines over multiple NLP tasks and datasets. |
Copied to clipboard
| Challenge: | Existing methods to fine-tune code intelligence models to individual tasks are costly and require large data sets. |
| Approach: | They propose a Transferable fine-tuning strategy for Code representation learning that uses a tunable prefix encoder to capture cross-task and cross-language transferable knowledge and apply it to downstream adaptation. |
| Outcome: | The proposed method can lead to superior performance on code-related tasks and encourage mutual reinforcement. |
Copied to clipboard
| Challenge: | Text-to-Image Synthesis (TIS) is a popular task to convert natural language texts into realistic images. |
| Approach: | They propose a transformer-based Chinese text-to-image synthesizer for high-resolution image generation that incorporates linguistic and relational knowledge facts into the model to ensure better performance without the usage of ultra-large models. |
| Outcome: | The proposed model outperforms existing models in Chinese with linguistic and relational knowledge facts. |
Copied to clipboard
| Challenge: | Existing studies have not noticed the safety risks of large language models . authors evaluated 1,400 questions in multi-turn dialogue coreference . |
| Approach: | They are the first to evaluate LLM safety in multi-turn dialogue coreference . they created a dataset of 1,400 questions and tested five open-source models . |
| Outcome: | The study shows that model safety decreases in multi-turn dialogue coreference scenarios . the highest success rate was with the LLaMA2-Chat-7b model, while the lowest was with mistral-7B-Instruct model . |
Copied to clipboard
| Challenge: | Recent studies have shown that Chain-of-Thought (CoT) prompting can be effective on complex reasoning tasks but generates unfaithful and unfactual reasoning chains. |
| Approach: | They propose a chain-of-knowledge prompting that elicits Large Language Models to generate explicit pieces of knowledge evidence in the form of structure triple. |
| Outcome: | The proposed method improves commonsense, factual, symbolic, and arithmetic reasoning tasks by estimating the reliability of the reasoning chains in terms of factuality and faithfulness. |
Copied to clipboard
| Challenge: | Existing methods for pre-trained language models rely on noisy data, which can be expensive if all parameters are updated. |
| Approach: | They propose a self-training framework that incorporates Monte Carlo dropouts into the model and judiciously selects reliable pseudo-labeled examples based on confidence and certainty. |
| Outcome: | The proposed framework improves performance and efficiency over multiple tasks over multiple datasets. |
Copied to clipboard
| Challenge: | Existing knowledge-enhanced pre-trained language models (PLMs) introduce redundant factual knowledge from knowledge bases and require complex modules. |
| Approach: | They propose a knowledge prompting-based PLM framework that incorporates factual knowledge into PLMs. |
| Outcome: | The proposed framework can be flexibly combined with existing mainstream PLMs. |
Copied to clipboard
| Challenge: | Existing studies focus on evaluating large language models in close-ended QA tasks, but many clinical decisions involve answering open-ended questions without pre-set options. |
| Approach: | They construct a benchmark to better understand large language models in the clinic . they use existing datasets to evaluate LLMs in clinical situations . |
| Outcome: | The proposed model outperforms human experts in multiple medical tasks. |
Copied to clipboard
| Challenge: | In multi-party chat, it is common for multiple conversations to occur concurrently . a new model that automatically disentangles conversation threads is proposed . |
| Approach: | They propose a Context-Aware Thread Detection model that automatically disentangles conversation threads in chat logs. |
| Outcome: | The proposed model outperforms state-of-the-art models on four real-world chat logs. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive progress in mathematical problem-solving . current approaches to enhance mathematical reasoning focus on instance-level modifications . |
| Approach: | They propose a framework that enhances mathematical reasoning through cross-problem instruction synthesis. |
| Outcome: | The proposed framework boosts mathematical reasoning by 18.0 points while maintaining high data efficiency. |
Copied to clipboard
| Challenge: | Long-context modeling is crucial for next-generation language models, but high computational cost of standard attention mechanisms poses significant computational challenges. |
| Approach: | They propose a natively trained Sparse Attention mechanism that integrates algorithms with hardware-aligned optimizations to achieve efficient long-context modeling. |
| Outcome: | The proposed model maintains or exceeds Full Attention models across general benchmarks, long-context tasks, and instruction-based reasoning. |
Copied to clipboard
| Challenge: | Extractive Question Answering (EQA) is one of the most essential tasks in Machine Reading Comprehension (MRC). |
| Approach: | They propose a framework that transforms extractive question answering into a non-autoregressive Masked Language Modeling (MLM) generation problem. |
| Outcome: | The proposed framework outperforms state-of-the-art approaches in few-shot learning scenarios by a large margin. |
Copied to clipboard
| Challenge: | Recent advances in visual-language pre-trained (VLP) models have greatly improved cross-modal retrieval performance . however, the fine-grained interactions between objects from different modalities are far from well-established . e-commerce domain lacks sufficient training data and fine-granular cross-modulal knowledge . |
| Approach: | They propose a visual-language pre-trained (VLP) image-text retrieval model that integrates cross-modal knowledge into the model to improve performance. |
| Outcome: | The proposed model improves performance on e-commerce image-text retrieval task by a large margin. |
Copied to clipboard
| Challenge: | Existing metrics based on text-level comparisons fail to assess the quality of captions produced by machines. |
| Approach: | They propose to use a machine-learned text-image grounding model to measure the accuracy of machine-generated captions and their correlation with human judgments. |
| Outcome: | The proposed metric has higher consistency with human judgments and is more accurate than existing metrics. |
Copied to clipboard
| Challenge: | Existing CodePTMs are mainly structure-free and structurebased, but how to fine-tune them remains a challenge. |
| Approach: | They propose a plug-and-play fine-tuning method that incorporates structural knowledge into pre-trained code models. |
| Outcome: | The proposed method can benefit CodePTMs more with limited training data. |
Copied to clipboard
| Challenge: | Multimodal large language models (MLLMs) have shown impressive capabilities in document understanding due to different types of annotation noise in training. |
| Approach: | They propose a method to reduce C&P knowledge conflicts across all tested MLLMs . they propose to use annotation noise to train models to understand document content . |
| Outcome: | The proposed method reduces C&P knowledge conflicts across all tested MLLMs and enhances their performance in both cognitive and perceptual tasks. |
Copied to clipboard
| Challenge: | Existing large language models (LLMs) can solve graph reasoning and generation tasks with parameter updates without sacrificing performance. |
| Approach: | They propose a structured format verbalizer to unify all graph data into a universal code-like format, which can simply represent the graph without any external graph-specific encoders. |
| Outcome: | The proposed framework outperforms GPT-4 and LLaMA2 in graph reasoning and generation tasks by more than 13% and 38%, respectively. |
Copied to clipboard
| Challenge: | Code pre-trained models have been proposed and widely applied in the domain of code intelligence. |
| Approach: | They propose a method that uses a plug-and-play graph neural network module as a tunable prefix to exploit structural information of source code. |
| Outcome: | The proposed method exploits structural information of source code and could replace full fine-tuning. |
Copied to clipboard
| Challenge: | Personalized news recommendation is an important technique for personalized news service. |
| Approach: | They propose to build a large-scale news recommendation dataset from Microsoft News . they demonstrate that news recommendation relies on the quality of news content understanding . |
| Outcome: | The proposed dataset contains 1 million users and more than 160k English news articles, each of which has rich textual content such as title, abstract and body. |
Copied to clipboard
| Challenge: | Existing code pre-trained models fail to consider inherent characteristics of codes . Existing methods to interpret code pretrained model fail to take into account inherent characteristics . |
| Approach: | They propose a probing method to quantitatively interpret how CodePTMs attend code structure. |
| Outcome: | The proposed method denoises input code sequences and measures commonality between token-level attention scores and pair-wise distances between corresponding AST nodes. |
Copied to clipboard
| Challenge: | Existing metrics for image captioning evaluation provide an overall quality score, which is difficult to infer specific description errors. |
| Approach: | They propose a fine-grained evaluation method REO for automatically measuring the performance of image captioning systems. |
| Outcome: | The proposed method achieves higher consistency with human judgments and provides more intuitive evaluation results than other metrics. |
Copied to clipboard
| Challenge: | Existing prompt-based NER models fail to detect entity boundaries, causing performance degradation. |
| Approach: | They propose a model which consists of a BART encoder and a parabiotic decoder and propose ' boundary expansion strategy' to enhance the model's capability in entity type classification. |
| Outcome: | The proposed model can achieve significant performance gains over state-of-the-art models. |
Copied to clipboard
| Challenge: | Large pre-trained language models (PLMs) are expensive and may not be open-sourced due to commercial considerations and potential risks of misuse. |
| Approach: | They propose to introduce gradient descent into black-box tuning scenario . they propose a method which integrates gradient descent and derivative-free optimization . |
| Outcome: | The proposed method achieves significant performance gains over previous state-of-the-art methods. |
Copied to clipboard
| Challenge: | Existing knowledge graphs focus on the representation and reasoning of general factual knowledge, while there are significant deficiencies in the understanding and reasoning for emotional knowledge. |
| Approach: | They propose a commonsense knowledge graph that can be used to represent emotional knowledge by combining theories from psychology, cognitive science, and linguistics. |
| Outcome: | The proposed model surpasses GPT-4-Turbo in the emotion-related tasks. |