Papers by Yuan-Fang Wang
Towards relation extraction from speech (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for extracting relations from speech have been neglected due to the nature of speech. |
| Approach: | They propose a listening information extraction task that uses speech to extract relation extraction from speech . they use a text-to-speech system and crowd-sourced native English speakers to test the task . |
| Outcome: | The proposed task extracts semantic relationships from speech data using a new model . the proposed task is more challenging than the existing method due to the characteristics of speech . |
Knowledge-enriched, Type-constrained and Grammar-guided Question Generation over Knowledge Bases (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for question generation over knowledge bases have low diversity and poor fluency due to the limited information contained in the subgraphs and semantic drift due to decoder’s oblivion of the semantics of the answer entity. |
| Approach: | They propose a knowledge-enriched, type-constrained and grammar-guided KBQG model that generates natural-language questions over a set of triples in the KB. |
| Outcome: | The proposed model outperforms existing methods on two widely-used benchmark datasets. |
On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex (2023.eacl-main)
Copied to clipboard
Terry Yue Zhuo, Zhuang Li, Yujin Huang, Fatemeh Shiri, Weiqing Wang, Gholamreza Haffari, Yuan-Fang Li
| Challenge: | Existing techniques for parsing natural-language utterances are vulnerable to adversarial attacks, requiring large amounts of labelled data and expensive human annotation. |
| Approach: | They propose to enhance the adversarial robustness of a prompt-based semantic parser based on a language model trained on code by constructing a set of demonstration examples. |
| Outcome: | The proposed method can be enhanced without significant amounts of labelled data or expensive human annotations on in-domain semantic parsing data. |
Rewarding What Matters: Step-by-Step Reinforcement Learning for Task-Oriented Dialogue (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing RL methods focus on generation tasks while neglecting dialogue state tracking (DST) for understanding. |
| Approach: | They propose a method that integrates RL into both understanding and generation tasks by introducing step-by-step rewards throughout the token generation. |
| Outcome: | The proposed approach achieves state-of-the-art results on three widely used datasets. |
Watch, Listen, and Describe: Globally and Locally Aligned Cross-Modal Attentions for Video Captioning (N18-2)
Copied to clipboard
| Challenge: | Existing multi-modal fusion methods have shown encouraging results in video understanding, but how to selectively fuse the multi-dimensional representations at different levels of details remains unexplored. |
| Approach: | They propose a hierarchically aligned cross-modal attention framework to fuse audio and visual cues at different levels of detail. |
| Outcome: | The proposed framework outperforms the previous best systems on the video captioning task. |
MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to update model-based indexes with new documents are expensive and require expensive retraining. |
| Approach: | They propose a framework that combines an expandable mixture of Low-Rank Adaptation experts with a layer-wise out-of-distribution-driven expansion strategy. |
| Outcome: | Experiments on NQ320k and MS MARCO Passage show that the proposed framework outperforms full-model update baselines with minimal parameter overhead and substantially lower training costs. |
No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling (P18-1)
Copied to clipboard
| Challenge: | Visual captioning is aimed at depicting the concrete content of images, but its capability of performing human-like understanding is still restrictive. |
| Approach: | They propose an Adversarial REward Learning framework to learn an implicit reward function from human demonstrations and optimize policy search with the learned reward function. |
| Outcome: | The proposed framework improves performance over state-of-the-art (SOTA) methods in cloning expert behaviors, but human evaluation shows that it achieves significant improvement in generating more human-like stories than SOTA systems. |