Papers by Chengwei Hu
Improving Continual Relation Extraction through Prototypical Contrastive Learning (2022.coling-1)
Copied to clipboard
| Challenge: | Continual relation extraction (CRE) aims to extract relations towards the continuous and iterative arrival of new data, of which the major challenge is the catastrophic forgetting of old tasks. |
| Approach: | They propose a Continual Relation Extraction framework with Contrastive Learning which is built with a classification network and a prototypical contrastive network to achieve incremental-class learning of CRE. |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on two public datasets and proves its effectiveness on improving performance. |
Improve Student’s Reasoning Generalizability through Cascading Decomposed CoTs Distillation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have tried to distill these capabilities into smaller language models (SLMs) however, these capabilities are often associated with more parameters, which is not practical to emergent in smaller models. |
| Approach: | They propose to decompose the traditional single-step learning process into two cascaded learning steps by restructuring the training objectives and concatenating the question with the rationale as input. |
| Outcome: | Extensive experiments show that the proposed method improves reasoning generalizability and diversity of the model. |
Capture the Key in Reasoning to Enhance CoT Distillation Generalization (2025.acl-long)
Copied to clipboard
| Challenge: | Existing distillation methods for Large Language Models (LLMs) focus on fine-tuning student SLMs on correct data, resulting in students struggling to learn the key instead of analyzing mistakes according to correct solutions. |
| Approach: | They propose a method that exposes key reasoning steps rather than simple fine-tuning students' CoTs data by using a set of prompts with similar reasoning paths but divergent conclusions. |
| Outcome: | The proposed method improves student SLMs' ability to learn key reasoning steps rather than fine-tuning them on teacher data. |
Data Augmentation using LLMs: Data Perspectives, Learning Paradigms and Challenges (2024.findings-acl)
Copied to clipboard
Bosheng Ding, Chengwei Qin, Ruochen Zhao, Tianze Luo, Xinze Li, Guizhen Chen, Wenhan Xia, Junjie Hu, Anh Tuan Luu, Shafiq Joty
| Challenge: | Data augmentation (DA) is a key technique for enhancing model performance by diversifying training examples without the need for additional data collection. |
| Approach: | They examine various strategies that utilize LLMs for data augmentation, including a novel exploration of learning paradigms where LLM-generated data is used for diverse forms of further training. |
| Outcome: | The proposed approach addresses the primary open challenges faced by LLMs in the field of large language models and aims to serve as a comprehensive guide for researchers and practitioners. |
Listen Again and Choose the Right Answer: A New Paradigm for Automatic Speech Recognition with Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have promoted generative error correction (GER) for automatic speech recognition (ASR). |
| Approach: | They propose a multimodal LLM to receive source speech as extra input and reformat it as a cloze test with logits calibration to remove input information redundancy and simplify GER with clear instructions. |
| Outcome: | The proposed model improves on 9 popular ASR datasets and is faster than vanilla GER. |
Refining Sample Embeddings with Relation Prototypes to Enhance Continual Relation Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to extract relationships from texts depend on memory size and replay these memorized samples in subsequent tasks. |
| Approach: | They propose to use a model to extract relations between entities from texts where the samples of different relations are delivered into the model continuously. |
| Outcome: | The proposed model outperforms the state-of-the-art models and avoids catastrophic forgetting. |
CT-GAT: Cross-Task Generative Adversarial Attack based on Transferability (2023.emnlp-main)
Copied to clipboard
| Challenge: | Neural network models are vulnerable to adversarial examples, and current methods based on adversarially transferable models rely on substitute models, which can be impractical and costly in real-world scenarios due to the unavailability of training data and the victim model’s structural details. |
| Approach: | They propose a novel approach that directly constructs adversarial examples by extracting transferable features across various tasks. |
| Outcome: | The proposed approach achieves superior attack performance with small cost on ten datasets and demonstrates that it is a novel approach. |
Beyond Output Matching: Bidirectional Alignment for Enhanced In-Context Learning (2025.acl-long)
Copied to clipboard
Chengwei Qin, Wenhan Xia, Fangkai Jiao, Chen Chen, Yuchen Hu, Bosheng Ding, Ruirui Chen, Shafiq Joty
| Challenge: | Existing methods to train student models on the generated outputs of teacher models are not efficient for ICL. |
| Approach: | They propose to align the output of smaller (student) models with that of larger (teacher) models by incorporating a ranking loss and aligning the token-level output distribution. |
| Outcome: | The proposed model outperforms baseline models on a variety of tasks involving language understanding, reasoning, and coding. |
AIR: Complex Instruction Generation via Automatic Iterative Refinement (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for generating complex instructions are resource-intensive and lack diversity. |
| Approach: | They propose a framework to generate complex instructions with constraints using a document-generated initial instruction and an iterative refinement framework to incorporate LLM-as-judge guidance. |
| Outcome: | The proposed framework significantly outperforms existing methods for generating complex instructions, and outperformed existing methods. |
Hearing Lips in Noise: Universal Viseme-Phoneme Mapping and Transfer for Robust Audio-Visual Speech Recognition (2023.acl-long)
Copied to clipboard
| Challenge: | Existing efforts to improve robustness of audio-visual speech recognition with visual information focus on audio modality . current approaches introduce noise adaptation techniques to improve reliability of AVSR task . |
| Approach: | They propose a visual-invariant modality to strengthen robustness of audio-visual speech recognition (AVSR) it can adapt to any testing noises without dependence on noisy training data, a.k.a., unsupervised noise adaptation. |
| Outcome: | The proposed method outperforms existing state-of-the-arts on visual speech recognition task under various noisy and clean conditions. |
Mitigating Out-of-Entity Errors in Named Entity Recognition: A Sentence-Level Strategy (2025.coling-main)
Copied to clipboard
| Challenge: | Existing models of named entity recognition (NER) suffer from the problem of Out-of-Entity (OOE), which hinders the achievement of satisfactory performance. |
| Approach: | They propose a framework which fully leverages sentence-level information to improve OOE-NER performance by exploiting pre-trained language models' ability to understand target entity’s sentence context with a template set and refines sentence representation based on positive and negative templates. |
| Outcome: | The proposed framework outperforms state-of-the-art models on five datasets on named entity recognition (NER) tasks. |
Relevant or Random: Can LLMs Truly Perform Analogical Reasoning? (2025.findings-acl)
Copied to clipboard
| Challenge: | Analogical reasoning is a unique ability of humans to address unfamiliar challenges by transferring strategies from relevant past experiences. |
| Approach: | They propose to use self-generated random examples to improve performance on a variety of reasoning tasks by incorporating relevant examples from relevant past experiences. |
| Outcome: | The proposed methods achieve comparable or even better performance on GSM8K with random biological examples. |
Chinese SimpleQA: A Chinese Factuality Evaluation for Large Language Models (2025.acl-long)
Copied to clipboard
Yancheng He, Shilong Li, Jiaheng Liu, Yingshui Tan, Weixun Wang, Hui Huang, Xingyuan Bu, Hangyu Guo, Chengwei Hu, Boren Zheng, Zhuoran Lin, Dekai Sun, Zhicheng Zheng, Wenbo Su, Bo Zheng
| Challenge: | Current frontier models sometimes generate false outputs or answers that are not substantiated by evidence. |
| Approach: | They propose Chinese SimpleQA, a Chinese benchmark to evaluate LLMs' factuality . they focus on Chinese language over 6 major topics with 99 diverse subtopics . |
| Outcome: | The Chinese SimpleQA benchmark evaluates the factuality ability of LLMs . the questions and answers are short and easy-to-evaluate . |
Overcoming Catastrophic Forgetting by Exemplar Selection in Task-oriented Dialogue System (2024.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that HESIT effectively alleviates catastrophic forgetting by exemplar selection, and achieves state-of-the-art performance on the largest CL benchmark of ToDs in terms of all metrics. |
| Approach: | They propose a method to overcome catastrophic forgetting in task-oriented dialogue systems by tracing their hyper-gradients and a retraining strategy that uses influential exemplars for periodic retrains. |
| Outcome: | The proposed method achieves state-of-the-art on the largest CL benchmark of ToDs in terms of all metrics. |