Papers by Xianwei Zhuang

Cyclical Contrastive Learning Based on Geodesic for Zero-shot Cross-lingual Spoken Language Understanding (2024.findings-acl)

Copied to clipboard

Xuxin Cheng, Zhihong Zhu, Bang Yang, Xianwei Zhuang, Hongxiang Li, Yuexian Zou

Semantic Text Processing

Challenge:	zero-shot cross-lingual SLU is a challenging task in low-resource languages . a lack of labeled training data makes it difficult to align representations of similar sentences .
Approach:	They propose a framework that uses cyclical contrastive learning to achieve consistency between languages . they propose to use geodesic to measure the similarity to construct positive and negative pairs .
Outcome:	The proposed framework achieves state-of-the-art performance on multiATIS++ and MTOP datasets.

Code-Switching Can be Better Aligners: Advancing Cross-Lingual SLU through Representation-Level and Prediction-Level Alignment (2024.acl-short)

Copied to clipboard

Zhihong Zhu, Xuxin Cheng, Zhanpeng Chen, Xianwei Zhuang, Zhiqi Huang, Yuexian Zou

Multilinguality

Challenge:	Existing code-switching-based cross-lingual spoken language understanding frameworks are limited to low-resource languages.
Approach:	They propose a cross-lingual spoken language understanding framework that leverages both code-switched and original sentences to achieve multi-level alignment.
Outcome:	The proposed framework can achieve multi-level alignment on two benchmarks across ten languages.

Can We Trust AI Doctors? A Survey of Medical Hallucination in Large Language and Large Vision-Language Models (2025.findings-acl)

Copied to clipboard

Zhihong Zhu, Yunyan Zhang, Xianwei Zhuang, Fan Zhang, Zhongwei Wan, Yuyan Chen, QingqingLong QingqingLong, Yefeng Zheng, Xian Wu

Multimodality

Challenge:	Hallucination is a critical challenge for large language models and large vision-language models (LVLMs) however, dedicated research on medical hallucinations remains unexplored.
Approach:	They provide a unified perspective on medical hallucination for both LLMs and LVLMs, and delve into its causes.
Outcome:	The proposed models have demonstrated impressive performance on a variety of medical benchmarks.

Dual-oriented Disentangled Network with Counterfactual Intervention for Multimodal Intent Detection (2024.emnlp-main)

Copied to clipboard

Zhanpeng Chen, Zhihong Zhu, Xianwei Zhuang, Zhiqi Huang, Yuexian Zou

Sentiment Analysis

Challenge:	Existing methods for multimodal intent detection have two limitations: (i) close entanglement of multimodal semantics with modal structures; (ii) insufficient learning of causal effects of semantic and modality-specific information on the final predictions.
Approach:	They propose a Dual-oriented Disentangled Network with Counterfactual Intervention model that decouples semantics-oriented and modality-oriented representations and a Counterfective Intervention Module that applies causal inference to understand causal effects by injecting confounders.
Outcome:	The proposed model overcomes key limitations in existing systems by effectively disentangling and utilizing modality-specific and multimodal semantic information.

MaCSC: Towards Multimodal-augmented Pre-trained Language Models via Conceptual Prototypes and Self-balancing Calibration (2024.naacl-long)

Copied to clipboard

Xianwei Zhuang, Zhichang Wang, Xuxin Cheng, Yuxin Xie, Liming Liang, Yuexian Zou

Multimodality

Challenge:	Existing approaches to training pre-trained language models (PLMs) focus on static image modality; inevitably encounter modality gaps and noise; and treat all modalities.
Approach:	They propose a multimodal-augmented framework that can infuse multimodal semantics into PLMs and facilitate a self-balancing calibration of information allocation.
Outcome:	The proposed framework outperforms baselines on multiple NLP tasks and outperformed existing frameworks.

What are the Generator Preferences for End-to-end Task-Oriented Dialog System? (2024.emnlp-main)

Copied to clipboard

Wanshi Xu, Xianwei Zhuang, Zhanpeng Chen, Zhihong Zhu, Xuxin Cheng, Yuexian Zou

Natural Language Interfaces

Challenge:	Existing methods to improve the accuracy of entity retrieval are not effective.
Approach:	They propose a framework that improves the performance of task-oriented dialogue systems by obtaining fine-grained matching information between contexts and entities and extracting the entity attribute shift matrix as preference signals.
Outcome:	The proposed framework outperforms existing methods and improves the quality of the dialogue.

PCAD: Towards ASR-Robust Spoken Language Understanding via Prototype Calibration and Asymmetric Decoupling (2024.acl-long)

Copied to clipboard

Xianwei Zhuang, Xuxin Cheng, Liming Liang, Yuxin Xie, Zhichang Wang, Zhiqi Huang, Yuexian Zou

Robustness in NLP

Challenge:	Spoken language understanding (SLU) suffers from error propagation from automatic speech recognition (ASR) in actual scenarios.
Approach:	They propose a framework which calibrates bias and errors and achieves adaptive-balanced decoupling training by a prototype-based loss model.
Outcome:	The proposed framework outperforms existing approaches and achieves state-of-the-art performance on three datasets.

Relevance Is a Guiding Light: Relevance-aware Adaptive Learning for End-to-end Task-oriented Dialogue System (2024.emnlp-main)

Copied to clipboard

Zhanpeng Chen, Zhihong Zhu, Wanshi Xu, Xianwei Zhuang, Yuexian Zou

Information Retrieval

Challenge:	Existing approaches to training task-oriented dialogue systems struggle with the Distractive Attributes Problem (DAP) Existing methods struggle to deal with false but similar knowledge (hard negative entities)
Approach:	They propose a two-stage training framework that eliminates hard negatives step-by-step and aligns retrieval with generation.
Outcome:	The proposed method eliminates hard negatives step-by-step and aligns retrieval with generation.

Game on Tree: Visual Hallucination Mitigation via Coarse-to-Fine View Tree and Game Theory (2024.emnlp-main)

Copied to clipboard

Xianwei Zhuang, Zhihong Zhu, Zhanpeng Chen, Yuxin Xie, Liming Liang, Yuexian Zou

Reasoning

Challenge:	Large vision-language models produce unfaithful visual hallucinations, also known as visual halluinations, which hinders their application in multimodal understanding and decision-making.
Approach:	They propose a plug-and-play train-free decoding algorithm for mitigating visual hallucinations . they leverage visual information to construct a coarse-to-fine visual view tree .
Outcome:	The proposed algorithm reduces visual hallucinations (VH) by leveraging visual information to construct a coarse-to-fine visual view tree (CFTree)

MoE-SLU: Towards ASR-Robust Spoken Language Understanding via Mixture-of-Experts (2024.findings-acl)

Copied to clipboard

Xuxin Cheng, Zhihong Zhu, Xianwei Zhuang, Zhanpeng Chen, Zhiqi Huang, Yuexian Zou

Robustness in NLP

Challenge:	Spoken language understanding (SLU) is a crucial task in task-oriented dialogue systems.
Approach:	They propose an ASR-Robust SLU framework based on the mixture-of-experts technique to generate additional transcripts from clean transcripts and use it to weigh the representations of the generated transcripts, ASR transcripts .
Outcome:	The proposed framework achieves state-of-the-art on three benchmark SLU datasets.

ATRI: Mitigating Multilingual Audio Text Retrieval Inconsistencies by Reducing Data Distribution Errors (2025.acl-long)

Copied to clipboard

Yuguo Yin, Yuxin Xie, Wenyuan Yang, Dongchao Yang, Jinghan Ru, Xianwei Zhuang, Liming Liang, Yuexian Zou

Semantic Text Processing

Challenge:	Existing multilingual audio-text retrieval schemes suffer from inconsistencies for instance similarity matching across languages.
Approach:	They propose a multilingual audio-text retrieval scheme that mitigates the impact of data distribution error on recall and consistency.
Outcome:	The proposed scheme achieves state-of-the-art performance on recall and consistency metrics for eight mainstream languages, including English.