Papers by Yuexian Zou
MRRL: Modifying the Reference via Reinforcement Learning for Non-Autoregressive Joint Multiple Intent Detection and Slot Filling (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing non-autoregressive models for multiple intent detection and slot filling have limited overall accuracy due to multi-modality problem and lack of alignment between correct predictions. |
| Approach: | They propose a method for multiple intent detection and slot filling that introduces a modifier and employs reinforcement learning to modify the reference. |
| Outcome: | The proposed method outperforms the previous best approach by 3.6 overall accuracy on MixATIS dataset. |
Cyclical Contrastive Learning Based on Geodesic for Zero-shot Cross-lingual Spoken Language Understanding (2024.findings-acl)
Copied to clipboard
| Challenge: | zero-shot cross-lingual SLU is a challenging task in low-resource languages . a lack of labeled training data makes it difficult to align representations of similar sentences . |
| Approach: | They propose a framework that uses cyclical contrastive learning to achieve consistency between languages . they propose to use geodesic to measure the similarity to construct positive and negative pairs . |
| Outcome: | The proposed framework achieves state-of-the-art performance on multiATIS++ and MTOP datasets. |
Code-Switching Can be Better Aligners: Advancing Cross-Lingual SLU through Representation-Level and Prediction-Level Alignment (2024.acl-short)
Copied to clipboard
| Challenge: | Existing code-switching-based cross-lingual spoken language understanding frameworks are limited to low-resource languages. |
| Approach: | They propose a cross-lingual spoken language understanding framework that leverages both code-switched and original sentences to achieve multi-level alignment. |
| Outcome: | The proposed framework can achieve multi-level alignment on two benchmarks across ten languages. |
Knowledge-enhanced Prompt Tuning for Dialogue-based Relation Extraction with Trigger and Label Semantic (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to determine semantic relation between two arguments in dialogues are limited due to the low information density of text. |
| Approach: | They propose a Knowledge-Enhanced Prompt-Tuning method to enhance DRE model by exploiting trigger and label semantics. |
| Outcome: | The proposed method achieves state-of-the-art in F1 and F1c scores on a DialogRE dataset. |
Learning to Match Representations is Better for End-to-End Task-Oriented Dialog System (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing systems for task-oriented dialogue lack belief states as supervisory signals. |
| Approach: | They propose a method for knowledge retrieval driven by matching representations . they use a matching signal extractor to extract matching representation between contexts and entities . |
| Outcome: | Experiments on three standard benchmarks show that the proposed method performs better than existing approaches. |
A Transformer-based Threshold-Free Framework for Multi-Intent NLU (2022.coling-1)
Copied to clipboard
| Challenge: | Existing models for multi-intent natural language understanding mainly detect multiple intents on threshold settings. |
| Approach: | They propose a transformer-based multi-intent NLU model with multi-task learning that exploits the information of the number of multiple intents in each utterance without additional manual annotations. |
| Outcome: | The proposed model achieves superior results on two public multi-intent datasets. |
Advancing General Multimodal Capability of Vision-language Models with Pyramid-descent Visual Position Encoding (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to encode visual positions inhibit the performance of vision-language Models (VLMs) however, language constitutes only one aspect of communication. |
| Approach: | They propose a method to assign visual position indexes from the periphery to the center and expand the central receptive field incrementally to enhance the perception of visual tokens within VLMs. |
| Outcome: | The proposed method reduces the relative distance between interrelated visual elements and instruction tokens, promoting a more rational allocation of attention weights and allowing for a multi-granularity perception of visual elements. |
Accelerating Multiple Intent Detection and Slot Filling via Targeted Knowledge Distillation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing non-autoregressive Spoken Language Understanding models suffer from multi-modality problem . current methods have little prior knowledge about the reference during inference . |
| Approach: | They propose a Targeted Knowledge Distillation Framework (TKDF) for multi-intent SLU that utilizes the knowledge distillation method to improve the performance. |
| Outcome: | The proposed model outperforms existing models on two public multi-intent datasets while speeding up by over 4.5 times. |
Dual-oriented Disentangled Network with Counterfactual Intervention for Multimodal Intent Detection (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for multimodal intent detection have two limitations: (i) close entanglement of multimodal semantics with modal structures; (ii) insufficient learning of causal effects of semantic and modality-specific information on the final predictions. |
| Approach: | They propose a Dual-oriented Disentangled Network with Counterfactual Intervention model that decouples semantics-oriented and modality-oriented representations and a Counterfective Intervention Module that applies causal inference to understand causal effects by injecting confounders. |
| Outcome: | The proposed model overcomes key limitations in existing systems by effectively disentangling and utilizing modality-specific and multimodal semantic information. |
MaCSC: Towards Multimodal-augmented Pre-trained Language Models via Conceptual Prototypes and Self-balancing Calibration (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to training pre-trained language models (PLMs) focus on static image modality; inevitably encounter modality gaps and noise; and treat all modalities. |
| Approach: | They propose a multimodal-augmented framework that can infuse multimodal semantics into PLMs and facilitate a self-balancing calibration of information allocation. |
| Outcome: | The proposed framework outperforms baselines on multiple NLP tasks and outperformed existing frameworks. |
Enhancing Code-Switching for Cross-lingual SLU: A Unified View of Semantic and Grammatical Coherence (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models rely on annotated training data, limiting their scalability to low-resource languages. |
| Approach: | They propose a method termed SoGo for zero-shot cross-lingual SLU that uses keywords as substitution options to extract keywords and a token-level alignment strategy to ensure grammatical coherence. |
| Outcome: | The proposed method improves zero-shot cross-lingual SLU across nine languages on MultiATIS++. |
ML-LMCL: Mutual Learning and Large-Margin Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Despite efforts to improve ASR robustness, errors from pipeline approaches can lead to error propagation. |
| Approach: | They propose a framework for improving ASR robustness in SLU by using mutual learning and large-margin contrastive learning. |
| Outcome: | The proposed framework outperforms existing models and achieves new state-of-the-art performance on three datasets. |
On Pursuit of Designing Multi-modal Transformer for Video Grounding (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for video grounding are not end-to-end, i.e., they rely on time-consuming post-processing steps to refine predictions. |
| Approach: | They propose an end-to-end multi-modal Transformer model that uses two encoders and a cross-modal decoder for grounding prediction. |
| Outcome: | The proposed model is 4.9% faster than existing models and is based on a set of encodings and decoders. |
Multimodal Prompt Learning for Product Title Generation with Extremely Limited Labels (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to generate informative titles for products with limited labels are inadequate for novel products. |
| Approach: | They propose a prompt-based approach to generate attractive titles for novel products . they use multimodal prompts to preserve characteristics and writing styles of novel products. |
| Outcome: | The proposed approach achieves state-of-the-art results on novel product categories with limited labels. |
Towards Unified Spoken Language Understanding Decoding via Label-aware Compact Linguistics Representations (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for intent detection and slot filling decoders could result in misaligned predictions for both tasks. |
| Approach: | They propose a method that leverages label embeddings to jointly guide the decoding process. |
| Outcome: | The proposed method outperforms existing methods on two single- and multi-intent SLU benchmarks and can be incorporated into existing models. |
What are the Generator Preferences for End-to-end Task-Oriented Dialog System? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve the accuracy of entity retrieval are not effective. |
| Approach: | They propose a framework that improves the performance of task-oriented dialogue systems by obtaining fine-grained matching information between contexts and entities and extracting the entity attribute shift matrix as preference signals. |
| Outcome: | The proposed framework outperforms existing methods and improves the quality of the dialogue. |
Federated Learning for Spoken Language Understanding (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods to improve robustness of models focus on a single dataset . but, there are few studies on how to combine merits of different datasets . |
| Approach: | They propose a federated learning framework that could unify datasets and tasks . they propose MV-Encoder as backbone of the framework to provide multi-granularity text representations . |
| Outcome: | The proposed framework improves on two SLU benchmark datasets and federated learning settings. |
PCAD: Towards ASR-Robust Spoken Language Understanding via Prototype Calibration and Asymmetric Decoupling (2024.acl-long)
Copied to clipboard
| Challenge: | Spoken language understanding (SLU) suffers from error propagation from automatic speech recognition (ASR) in actual scenarios. |
| Approach: | They propose a framework which calibrates bias and errors and achieves adaptive-balanced decoupling training by a prototype-based loss model. |
| Outcome: | The proposed framework outperforms existing approaches and achieves state-of-the-art performance on three datasets. |
Towards Multi-modal Sarcasm Detection via Disentangled Multi-grained Multi-modal Distilling (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to sarcasm detection focus on textual and intra-modal incongruity . mainstream approaches process input of each modality in a holistic manner, resulting in redundant and unrefined information. |
| Approach: | They propose a framework for multi-modal sarcasm detection that disentangles modality representations into latent spaces and conducts multi-grained knowledge distilling. |
| Outcome: | The proposed framework overpowers existing methods on a common benchmark. |
Relevance Is a Guiding Light: Relevance-aware Adaptive Learning for End-to-end Task-oriented Dialogue System (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to training task-oriented dialogue systems struggle with the Distractive Attributes Problem (DAP) Existing methods struggle to deal with false but similar knowledge (hard negative entities) |
| Approach: | They propose a two-stage training framework that eliminates hard negatives step-by-step and aligns retrieval with generation. |
| Outcome: | The proposed method eliminates hard negatives step-by-step and aligns retrieval with generation. |
MultiCapCLIP: Auto-Encoding Prompts for Zero-Shot Multilingual Visual Captioning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for supervised visual captioning require large scale of images or videos paired with descriptions in a specific language. |
| Approach: | They propose a zero-shot approach that generates captions for different scenarios without labeling . they use concept prompts to retrieve concepts and auto-encode them to learn writing styles . |
| Outcome: | The proposed approach generates captions for different scenarios and languages without labeled vision-caption pairs. |
Game on Tree: Visual Hallucination Mitigation via Coarse-to-Fine View Tree and Game Theory (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large vision-language models produce unfaithful visual hallucinations, also known as visual halluinations, which hinders their application in multimodal understanding and decision-making. |
| Approach: | They propose a plug-and-play train-free decoding algorithm for mitigating visual hallucinations . they leverage visual information to construct a coarse-to-fine visual view tree . |
| Outcome: | The proposed algorithm reduces visual hallucinations (VH) by leveraging visual information to construct a coarse-to-fine visual view tree (CFTree) |
MoE-SLU: Towards ASR-Robust Spoken Language Understanding via Mixture-of-Experts (2024.findings-acl)
Copied to clipboard
| Challenge: | Spoken language understanding (SLU) is a crucial task in task-oriented dialogue systems. |
| Approach: | They propose an ASR-Robust SLU framework based on the mixture-of-experts technique to generate additional transcripts from clean transcripts and use it to weigh the representations of the generated transcripts, ASR transcripts . |
| Outcome: | The proposed framework achieves state-of-the-art on three benchmark SLU datasets. |
ATRI: Mitigating Multilingual Audio Text Retrieval Inconsistencies by Reducing Data Distribution Errors (2025.acl-long)
Copied to clipboard
Yuguo Yin, Yuxin Xie, Wenyuan Yang, Dongchao Yang, Jinghan Ru, Xianwei Zhuang, Liming Liang, Yuexian Zou
| Challenge: | Existing multilingual audio-text retrieval schemes suffer from inconsistencies for instance similarity matching across languages. |
| Approach: | They propose a multilingual audio-text retrieval scheme that mitigates the impact of data distribution error on recall and consistency. |
| Outcome: | The proposed scheme achieves state-of-the-art performance on recall and consistency metrics for eight mainstream languages, including English. |
Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answering (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that our model achieves state-of-the-art results on three SQA benchmarks. |
| Approach: | They propose a self-supervised training stage and a contrastive representation learning stage for spoken question answering with auxiliary tasks and augmentation strategies. |
| Outcome: | The proposed model achieves state-of-the-art results on three SQA benchmarks. |
End-to-end Spoken Conversational Question Answering: Task, Dataset and Model (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for conversational question answering significantly degrade on datasets . a new task aims to enable systems to model complex dialogues flow given the speech documents . |
| Approach: | They propose a new Spoken Conversational Question Answering task to model human conversations . they propose DDNet, which ingests cross-modal information to achieve fine-grained representations of speech and language modalities. |
| Outcome: | The proposed method achieves superior performance in spoken conversational question answering. |
Soul-Mix: Enhancing Multimodal Machine Translation with Manifold Mixup (2024.acl-long)
Copied to clipboard
| Challenge: | Multimodal machine translation (MMT) aims to improve the performance of machine translation with the help of visual information. |
| Approach: | They propose a multimodal machine translation mixup method that integrates visual information into conventional text-only neural machine translation systems. |
| Outcome: | The proposed method outperforms existing models on a multi-directional dataset with fewer parameters and achieves new state-of-the-art performance. |
Rethinking Skip Connection with Layer Normalization (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods to solve the optimization problem of deep neural networks are not linear, but can be used as a modulating mechanism between the input and output. |
| Approach: | They propose to use skip connection to adjust the scale of the input and output to improve the performance. |
| Outcome: | The proposed approach improves performance and convergence of deep neural networks and can be applied to machine translation and image classification datasets. |