Papers by Liming Wang
Mitigating the Inconsistency Between Word Saliency and Model Confidence with Pathological Contrastive Training (2022.findings-acl)
Copied to clipboard
| Challenge: | Neural networks are used for various NLP tasks, but their complexity makes them difficult to interpret. |
| Approach: | They propose a framework to mitigate the model pathology and obtain more interpretable models by using contrastive learning and saliency-based samples augmentation to calibrate the sentences representation. |
| Outcome: | The proposed framework can mitigate the model pathology and generate more interpretable models while keeping the model performance. |
Listen, Decipher and Sign: Toward Unsupervised Speech-to-Sign Language Recognition (2023.findings-acl)
Copied to clipboard
Liming Wang, Junrui Ni, Heting Gao, Jialu Li, Kai Chieh Chang, Xulin Fan, Junkai Wu, Mark Hasegawa-Johnson, Chang Yoo
| Challenge: | Existing supervised sign language recognition systems rely on well-annotated data . instead, an unsupervised speech-to-sign language recognition system learns to translate between spoken and sign languages by observing only non-parallel speech and sign-language corpora. |
| Approach: | They propose an unsupervised speech-to-sign language recognition system that can translate between spoken and sign languages by observing only non-parallel speech and sign-language corpora. |
| Outcome: | The proposed approach outperforms baseline models on sign language corpora by 50% . the proposed approach is available at https://github.com/cactuswiththoughts/UnsupSpeech2Sign.git . |
Self-supervised Semantic-driven Phoneme Discovery for Zero-resource Speech Recognition (2022.acl-long)
Copied to clipboard
| Challenge: | a new approach for self-supervised speech representation learning is proposed . a phoneme inventory learning model is based on a discrete representation of speech . |
| Approach: | They propose a neural discrete representation learning model for self-supervised phoneme inventory learning with raw speech and word labels. |
| Outcome: | The proposed model learns better phoneme-level representations and lowers error rates on TIMIT and Mboshi benchmarks than previous state-of-the-art models. |
Multimodal Fusion with Co-Attention Networks for Fake News Detection (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to detect fake news with textual and visual contents are ineffective because they concatenate unimodal features without considering inter-modality relations. |
| Approach: | They propose to fuse textual and visual features for fake news detection using multimodal co-attention networks to learn inter-dependencies between multimodal features. |
| Outcome: | Extensive experiments on two realworld datasets show that the proposed network outperforms state-of-the-art methods and learns inter-dependencies among multimodal features. |
MaCSC: Towards Multimodal-augmented Pre-trained Language Models via Conceptual Prototypes and Self-balancing Calibration (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to training pre-trained language models (PLMs) focus on static image modality; inevitably encounter modality gaps and noise; and treat all modalities. |
| Approach: | They propose a multimodal-augmented framework that can infuse multimodal semantics into PLMs and facilitate a self-balancing calibration of information allocation. |
| Outcome: | The proposed framework outperforms baselines on multiple NLP tasks and outperformed existing frameworks. |
Similarizing the Influence of Words with Contrastive Learning to Defend Word-level Adversarial Text Attack (2023.findings-acl)
Copied to clipboard
| Challenge: | Neural language models are vulnerable to word-level adversarial text attacks . previous word-based search methods assume important words influence prediction . |
| Approach: | They propose a method for similarizing the influence of words with contrast learning that encourages model to learn sentence representations in which words of varying importance have a more uniform influence on prediction. |
| Outcome: | The proposed method is compatible with various training methods and improves model robustness against various adversarial attacks. |
ReLearn: Unlearning via Learning for Large Language Models (2025.acl-long)
Copied to clipboard
Haoming Xu, Ningyuan Zhao, Liming Yang, Sendong Zhao, Shumin Deng, Mengru Wang, Bryan Hooi, Nay Oo, Huajun Chen, Ningyu Zhang
| Challenge: | Existing methods for unlearning large language models often rely on reverse optimization to reduce target token probabilities. |
| Approach: | They propose a data augmentation and fine-tuning pipeline for effective unlearning . they propose augmentation, evaluation frameworks to measure contextual forgetting . |
| Outcome: | The proposed framework achieves targeted forgetting while preserving high-quality outputs. |
PCAD: Towards ASR-Robust Spoken Language Understanding via Prototype Calibration and Asymmetric Decoupling (2024.acl-long)
Copied to clipboard
| Challenge: | Spoken language understanding (SLU) suffers from error propagation from automatic speech recognition (ASR) in actual scenarios. |
| Approach: | They propose a framework which calibrates bias and errors and achieves adaptive-balanced decoupling training by a prototype-based loss model. |
| Outcome: | The proposed framework outperforms existing approaches and achieves state-of-the-art performance on three datasets. |
A Theory of Unsupervised Speech Recognition (2023.acl-long)
Copied to clipboard
| Challenge: | Unsupervised speech recognition (ASR) is the problem of learning automatic speech recognition systems from unpaired speech-only and text-only corpora. |
| Approach: | They propose a general theoretical framework to study the properties of pasted macro ‘ASRU’/ systems based on random matrix theory and the theory of neural tangent kernels. |
| Outcome: | The proposed framework proves various learnability conditions and sample complexity bounds on synthetic languages with three classes of transition graphs. |
MMAPG: A Training-Free Framework for Multimodal Multi-hop Question Answering via Adaptive Planning Graphs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing multimodal question answering models rely on sequential retrieval and reasoning, but this single-path paradigm makes them vulnerable to errors due to misleading intermediate steps. |
| Approach: | They propose a multimodal multi-hop question answering framework guided by an Adaptive Planning Graph . they propose modality-specific strategies that dynamically adapt to distinct data types . |
| Outcome: | The proposed framework outperforms existing models that rely on training. |
PARSE: An Efficient Search Method for Black-box Adversarial Text Attacks (2022.coling-1)
Copied to clipboard
| Challenge: | Neural networks are vulnerable to adversarial examples, i.e., under a black-box scenario. |
| Approach: | They propose a word-level search algorithm that searches for subareas under dynamic search space following the subarea importance. |
| Outcome: | The proposed algorithm can achieve comparable success rates to complex search methods while saving numerous queries and time. |
HydraRAG: Structured Cross-Source Enhanced Large Language Model Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current RAG system retrieves evidence from knowledge graphs and text documents but has limitations in multi-hop reasoning, multi-entity questions, and source verification. |
| Approach: | They propose a training-free framework that unifies graph topology, document semantics, and source reliability to support deep, faithful reasoning in large language models. |
| Outcome: | The proposed framework outperforms the current hybrid model-based model-driven system by 20.3% and 30.1% on seven benchmark datasets. |
Rethinking Word-level Adversarial Attack: The Trade-off between Efficiency, Effectiveness, and Imperceptibility (2024.lrec-main)
Copied to clipboard
| Challenge: | Neural language models have demonstrated impressive performance but remain vulnerable to word-level adversarial attacks. |
| Approach: | They propose two standardized search spaces to address the problem of word-level adversarial attacks. |
| Outcome: | The proposed search spaces improve performance and trade-offs in different scenarios. |
Contrastive Learning with Adversarial Examples for Alleviating Pathology of Language Model (2023.acl-long)
Copied to clipboard
| Challenge: | Existing interpretation methods fail to obtain faithful attributions on these models, thereby failing to reveal potential flaws and biases. |
| Approach: | They propose a Contrastive learning regularization method which calibrates the sentence representation of out-of-distribution examples and utilizes adversarial examples to introduce direction information in regularization. |
| Outcome: | The proposed method alleviates the model pathology while impacting generalization ability on in-distribution examples and thus helps interpretation methods obtain more faithful results. |