Papers by Haitao Zheng
Integrating User History into Heterogeneous Graph for Dialogue Act Recognition (2020.coling-main)
Copied to clipboard
| Challenge: | Existing models cannot fully recognize the specific expressions given by users due to the informality and diversity of natural language expressions. |
| Approach: | They propose a Heterogeneous User History graph convolution network which utilizes the user’s historical answers grouped by DA labels as additional clues to recognize the DA label of utterances. |
| Outcome: | The proposed model outperforms the state-of-the-art methods on two benchmark datasets and shows that it integrates user’s historical answers. |
OpenPrompt: An Open-source Framework for Prompt-learning (2022.acl-demo)
Copied to clipboard
| Challenge: | Prompt-learning is a new paradigm in natural language processing, adapting pre-trained language models to cloze-style prediction, autoregressive modeling, or sequence to sequence generation. |
| Approach: | They propose a framework for prompt-learning that integrates pre-trained language models with a unified framework. |
| Outcome: | The proposed framework is easy to use and flexible enough to integrate with other frameworks. |
CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding (2021.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models are vulnerable to simple perturbations, causing poor robustness . recent studies show that adversarial training is useless or harmful for the model to detect these semantic changes. |
| Approach: | They propose to use adversarial training to improve the robustness of pre-trained models . they propose to construct negative examples with similar and opposite semantics . |
| Outcome: | Empirical results show that the proposed approach improves on sentiment analysis, reasoning, and reading comprehension tasks. |
Few-shot Classification with Hypersphere Modeling of Prototypes (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fewshot learning use embeddings in space, but they lack expressivity and are difficult to perform statistically. |
| Approach: | They propose a method where class information is represented by hyperspheres with dynamic sizes with two sets of learnable parameters: the hypersphere’s center and the radius. |
| Outcome: | The proposed method is much more expressive than embeddings and performs better than statistical modeling. |
Exploring Lottery Prompts for Pre-trained Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to optimize pre-trained language models are expensive and slow to scale. |
| Approach: | They propose to search for instance-level lottery prompts and generalize them to unseen data . they validate the assumption that for every instance, there is almost always a lottery prompt that induces the correct prediction from the PLM . |
| Outcome: | The proposed method can achieve comparable results with other gradient-free and optimization-free baselines. |
Wasserstein Selective Transfer Learning for Cross-domain Text Mining (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve the learning of data-scarce target domains have negative transfer due to the data distributions between source and target domain. |
| Approach: | They propose a method that uses a reinforced selector to select helpful data for transfer learning and a Wasserstein-based discriminator to maximize the distance between the selected data and target data. |
| Outcome: | The proposed method performs better on three real-world text mining tasks. |
Few-NERD: A Few-shot Named Entity Recognition Dataset (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to few-shot named entity recognition (NER) focus on coarse-grained entities with few examples, while most unseen entities are fine-grounded. |
| Approach: | They present a human-annotated few-shot named entity recognition dataset . they construct benchmark tasks to assess the generalization capability of models . |
| Outcome: | The proposed model is the first few-shot NER dataset and the largest human-crafted NER data set. |
R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models with stacked layers do not explicitly model hierarchical structure of language understanding. |
| Approach: | They propose a recursive Transformer model based on differentiable CKY style binary trees to emulate hierarchical composition process. |
| Outcome: | The proposed model can predict words given their left and right abstraction nodes. |
Answer-driven Deep Question Generation based on Reinforcement Learning (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for deep question generation focus on enhancing document representations, but little attention is paid to the answer information. |
| Approach: | They propose a deep question generation model that makes better use of the target answer as a guidance to facilitate question generation. |
| Outcome: | The proposed model outperforms state-of-the-art models in automatic and human evaluations on the hotpotQA dataset. |
Summarize before Aggregate: A Global-to-local Heterogeneous Graph Inference Network for Conversational Emotion Recognition (2020.coling-main)
Copied to clipboard
| Challenge: | Existing studies focus on modeling emotion influences with utterance-level features, with little attention paid on phrase-level semantic connection between utterrances. |
| Approach: | They propose a two-stage Summarization and Aggregation Graph Inference Network which integrates inference for topic-related emotional phrases and local dependency reasoning over neighbouring utterances in a global-to-local fashion. |
| Outcome: | The proposed model outperforms the state-of-the-art models on three CER benchmark datasets. |
The World is Not Binary: Learning to Rank with Grayscale Data for Dialogue Response Selection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to learning-to-rank response selection are suboptimal due to ignorance of diversity of response quality. |
| Approach: | They propose to use off-the-shelf response retrieval models as automatic grayscale data generators to train response selection models. |
| Outcome: | The proposed approach can be automated without human effort on grayscale data. |
Chinese Relation Extraction with Multi-Grained Information and External Linguistic Knowledge (P19-1)
Copied to clipboard
| Challenge: | Existing methods for Chinese relation extraction suffer from segmentation errors and ambiguity of polysemy. |
| Approach: | They propose a multi-grained lattice framework for Chinese relation extraction . they incorporate word-level information into character sequence inputs to avoid segmentation errors . |
| Outcome: | The proposed model outperforms existing models on three real-world datasets in distinct domains. |
Linguistic Rules-Based Corpus Generation for Native Chinese Grammatical Error Correction (2022.findings-emnlp)
Copied to clipboard
Shirong Ma, Yinghui Li, Rongyi Sun, Qingyu Zhou, Shulin Huang, Ding Zhang, Li Yangning, Ruiyang Liu, Zhongli Li, Yunbo Cao, Haitao Zheng, Ying Shen
| Challenge: | Chinese Grammatical Error Correction (CGEC) is a challenging NLP task and a common application in human daily life. |
| Approach: | They propose a linguistic rules-based approach to construct large-scale CGEC training corpora with automatically generated grammatical errors. |
| Outcome: | The proposed method improves performance of existing CGEC models and the benchmark is excellent resource for further development. |
Learning from the Dictionary: Heterogeneous Knowledge Guided Fine-tuning for Chinese Spell Checking (2022.findings-emnlp)
Copied to clipboard
Yinghui Li, Shirong Ma, Qingyu Zhou, Zhongli Li, Li Yangning, Shulin Huang, Ruiyang Liu, Chao Li, Yunbo Cao, Haitao Zheng
| Challenge: | Chinese Spell Checking (CSC) aims to detect and correct Chinese spelling errors. |
| Approach: | They propose a framework which renders Chinese Spell Checking model to learn heterogeneous knowledge from the dictionary in terms of phonetics, vision, and meaning. |
| Outcome: | The proposed framework renders the CSC model to learn heterogeneous knowledge from the dictionary in terms of phonetics, vision, and meaning. |
Prompt-learning for Fine-grained Entity Typing (2022.findings-emnlp)
Copied to clipboard
Ning Ding, Yulin Chen, Xu Han, Guangwei Xu, Xiaobin Wang, Pengjun Xie, Haitao Zheng, Zhiyuan Liu, Juanzi Li, Hong-Gee Kim
| Challenge: | Extensive experiments on fine-grained entity typing under fully supervised, few-shot, and zero-shot settings show the effectiveness of prompt-learning. |
| Approach: | They propose a prompt-learning pipeline that stimulates versatile knowledge of pre-trained language models (PLMs) by constructing entity-oriented verbalizers and templates and conducting masked language modeling. |
| Outcome: | The proposed approach can be applied to fine-grained entity typing in fully supervised, few-shot, and zero-shot scenarios. |
Let LLMs Take on the Latest Challenges! A Chinese Dynamic Question Answering Benchmark (2025.coling-main)
Copied to clipboard
Zhikun Xu, Yinghui Li, Ruixue Ding, Xinyu Wang, Boli Chen, Yong Jiang, Haitao Zheng, Wenlian Lu, Pengjun Xie, Fei Huang
| Challenge: | Recent work has noted that due to the extremely high cost of iterative updates of LLMs, they are often unable to answer dynamic questions well. |
| Approach: | They propose a Chinese Dynamic QA benchmark containing question-answer pairs related to the latest dynamic questions on the Chinese Internet. |
| Outcome: | The proposed benchmark will be one of the key data resources for improving LLMs’ Chinese question-answering ability in the future. |
Event Detection with Trigger-Aware Lattice Neural Network (D19-1)
Copied to clipboard
| Challenge: | Event detection is a key part of event extraction, but there are two issues with word-based models in languages without natural delimiters, such as Chinese. |
| Approach: | They propose a framework that can solve the problem of word- trigger mismatch . they also use an external knowledge base to model polysemous characters and words . |
| Outcome: | The proposed model outperforms state-of-the-art methods on two benchmark datasets and outperformed previous state- of-the art methods significantly. |
LatEval: An Interactive LLMs Evaluation Benchmark with Incomplete Information from Lateral Thinking Puzzles (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks, such as MMLU, C-Eval, and GSM8K, evaluate models by posing a variety of problems, including problems about mathematics, science, law, and general knowledge. |
| Approach: | They propose a benchmark which assesses the model’s lateral thinking within an interactive framework. |
| Outcome: | The proposed evaluation benchmark assesses the model’s lateral thinking within an interactive framework. |
Coupling Distant Annotation and Adversarial Training for Cross-Domain Chinese Word Segmentation (2020.acl-main)
Copied to clipboard
| Challenge: | Fully supervised neural approaches have achieved significant progress in the task of Chinese word segmentation (CWS) however, they suffer from the cross-domain issue when they come to processing of out-of-domain data. |
| Approach: | They propose to use Chinese word as a target domain for distant annotation and adversarial training to reduce noise and maximize utilization of the source domain information. |
| Outcome: | The proposed method outperforms existing state-of-the-art methods on real-world datasets and significantly outperformed previous state- of-the art methods. |
Social-aware Sparse Attention Network for Session-based Social Recommendation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for predicting the next item for an anonymous session do not capture user preferences and noisy irrelevant interactions. |
| Approach: | They propose to use social networks and historical sessions to provide personalized recommendations for the current session. |
| Outcome: | The proposed model outperforms existing models on two benchmark datasets. |
Fusion or Defusion? Flexible Vision-and-Language Pre-Training (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to vision-and-language pretraining (VLP) lack effectiveness and efficiency in downstream multimodal tasks. |
| Approach: | They propose a flexible vision-and-language pre-training model by incorporating cross-modal fusions into a dual-encoder architecture and a cross-module knowledge transfer strategy to guide the training process. |
| Outcome: | The proposed model is well-equipped with effectiveness and efficiency compared with other strong VLP models. |