Papers by Pascale Fung

Reducing Gender Bias in Abusive Language Detection (D18-1)

Challenge:	Using a script generation system, scriptwriters can customize their scripts using video retrieval.
Approach:	They propose a controllable pipeline that generates complete scripts, including dialogues and scene descriptions, and presents visually using video retrieval.
Outcome:	The proposed system outperforms baselines on both automatic and human evaluations, especially in genre control.

Copied to clipboard

Ji Ho Park, Jamin Shin, Pascale Fung

Representation Learning

Challenge:	Abusive language detection models tend to be biased toward identity words of a certain group of people . recent studies have raised concerns about the robustness of such systems .
Approach:	They propose to use debiased word embeddings, gender swap data augmentation to reduce model bias . they also propose to fine-tune models with a larger corpus to correct such bias if needed .
Outcome:	The proposed methods reduce model bias by 90-98% and can be extended to correct model bias in other scenarios.

Automatic Speech Recognition Datasets in Cantonese: A Survey and New Dataset (2022.lrec-1)

Copied to clipboard

Tiezheng Yu, Rita Frieske, Peng Xu, Samuel Cahyawijaya, Cheuk Tung Yiu, Holy Lovenia, Wenliang Dai, Elham J. Barezi, Qifeng Chen, Xiaojuan Ma, Bertram Shi, Pascale Fung

MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models (2020.emnlp-main)

Challenge:	In this paper, we address the problem of data scarcity for the Hong Kong Cantonese language . due to the popularization of deep learning, ASR technology has led to a significant improvement in recognizing many languages.
Approach:	They propose to use a dataset to analyze the data available for the Hong Kong Cantonese language . they use zh-HK as a source and a state-of-the-art ASR model to build a powerful model .
Outcome:	The proposed model improves on the biggest existing dataset, Common Voice zh-HK.

Copied to clipboard

Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Raul Puri, Pascale Fung, Anima Anandkumar, Bryan Catanzaro

Personalizing Dialogue Agents via Meta-Learning (P19-1)

Challenge:	Existing pre-trained large language models have shown unparalleled generative capabilities, but they are not controllable.
Approach:	They propose a framework that uses large-scale language models and adds control to text generation by incorporating an external knowledge base.
Outcome:	The proposed model generates more fluent, consistent, and coherent stories with less repetition and higher diversity compared to previous work on the ROC story dataset.

Copied to clipboard

Andrea Madotto, Zhaojiang Lin, Chien-Sheng Wu, Pascale Fung

Korean Language Modeling via Syntactic Guide (2022.lrec-1)

Challenge:	Existing personalized dialogue models use human designed persona descriptions to improve dialogue consistency.
Approach:	They propose to extend Model-Agnostic Meta-Learning (MAML) to personalized dialogue learning without using persona descriptions.
Outcome:	The proposed model outperforms baseline models in terms of human-evaluated fluency and consistency on a persona-chat dataset.

Copied to clipboard

Hyeondey Kim, Seonhoon Kim, Inho Kang, Nojun Kwak, Pascale Fung

What Makes for Good Image Captions? (2025.findings-emnlp)

Challenge:	Existing research on pre-trained language models focuses on widely-used languages . however, not every language can benefit from such models due to computational resources .
Approach:	They propose to build a pre-trained language model that understands the linguistic phenomena in the target language with low resources.
Outcome:	The proposed model improves the performance of Korean language understanding tasks.

Copied to clipboard

Delong Chen, Samuel Cahyawijaya, Etsuko Ishii, Ho Shu Chan, Yejin Bang, Pascale Fung

Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems (P18-1)

Challenge:	a formal information-theoretic framework is developed for image captioning . the pyramid of captions is a method that generates enriched captions by integrating local and global visual information.
Approach:	They propose a formal information-theoretic framework for image captioning . they propose 'Pyramid of Captions' method that generates enriched captions .
Outcome:	The proposed framework provides a flexible foundation for analyzing and optimizing image captioning systems across diverse task requirements.

Copied to clipboard

Andrea Madotto, Chien-Sheng Wu, Pascale Fung

AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization (2021.naacl-main)

Challenge:	End-to-end task-oriented dialog systems often suffer from the challenge of incorporating knowledge bases.
Approach:	They propose a novel yet simple end-to-end differentiable model called memory-tosequence to address this issue.
Outcome:	The proposed model can be trained faster and achieve state-of-the-art performance on three different task-oriented dialog datasets.

Copied to clipboard

Tiezheng Yu, Zihan Liu, Pascale Fung

Low-Resource NLP

Challenge:	State-of-the-art abstractive summarization models rely on extensive labeled data, which lowers their generalization ability on domains where such data are not available.
Approach:	They propose to use domain adaptation methods to simulate the low-resource domain adaptation setting for abstractive summarization systems with existing datasets across six diverse target domains.
Outcome:	The proposed model can be used to adapt to a low-resource domain adaptation setting.

Zero-Shot Dialogue State Tracking via Cross-Task Transfer (2021.emnlp-main)

Copied to clipboard

Zhaojiang Lin, Bing Liu, Andrea Madotto, Seungwhan Moon, Zhenpeng Zhou, Paul Crook, Zhiguang Wang, Zhou Yu, Eunjoon Cho, Rajen Subba, Pascale Fung

MoEL: Mixture of Empathetic Listeners (D19-1)

Challenge:	Existing approaches to training a dialogue state tracking model require extensive annotated dialogue data.
Approach:	They propose to transfer cross-task knowledge from general question answering corpora to QA model that can handle zero-shot DST.
Outcome:	The proposed model improves existing zero-shot and few-shot results on MultiWoz and shows better generalization ability in unseen domains.

Copied to clipboard

Zhaojiang Lin, Andrea Madotto, Jamin Shin, Peng Xu, Pascale Fung

Context Generation Improves Open Domain Question Answering (2023.findings-eacl)

Challenge:	Neural network approaches for conversation models have shown to be successful in generating fluent and relevant responses.
Approach:	They propose a novel end-to-end approach for modeling empathy in dialogue systems by using Mixture of Empathetic Listeners (MoEL).
Outcome:	The proposed model outperforms multitask training baseline in terms of empathy, relevance, and fluency.

Copied to clipboard

Dan Su, Mostofa Patwary, Shrimai Prabhumoye, Peng Xu, Ryan Prenger, Mohammad Shoeybi, Pascale Fung, Anima Anandkumar, Bryan Catanzaro

NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages (2023.eacl-main)

Challenge:	Existing closed-book question answering methods do not fully exploit the parameterized knowledge.
Approach:	They propose a closed-book QA framework which uses a coarse-to-fine approach to extract the relevant knowledge and answer a question.
Outcome:	The proposed method outperforms open-book QA methods on three QA benchmarks.

Copied to clipboard

Genta Indra Winata, Alham Fikri Aji, Samuel Cahyawijaya, Rahmad Mahendra, Fajri Koto, Ade Romadhony, Kemal Kurniawan, David Moeljadi, Radityo Eko Prasojo, Pascale Fung, Timothy Baldwin, Jey Han Lau, Rico Sennrich, Sebastian Ruder

Multilinguality

Challenge:	In Indonesia, many languages are endangered and some are even extinct due to the unavailability of data resources and benchmarks.
Approach:	They propose a high-quality multilingual parallel corpus that covers 10 local languages from Indonesia.
Outcome:	The proposed resource includes sentiment and machine translation datasets, and bilingual lexicons.

Multi-hop Question Generation with Graph Convolutional Network (2020.findings-emnlp)

Copied to clipboard

Dan Su, Yan Xu, Wenliang Dai, Ziwei Ji, Tiezheng Yu, Pascale Fung

Text Generation

Challenge:	Existing studies on text-based QG focus on generating SQuAD-style questions.
Approach:	They propose a multi-hop question generation model that does context encoding in multiple hops with Graph Convolutional Network and encoder fusion via an Encoder Reasoning Gate.
Outcome:	Empirical results show that the proposed model generates fluent questions with high completeness and outperforms baselines on automatic evaluation metrics.

Mitigating Framing Bias with Polarity Minimization Loss (2023.findings-emnlp)

Copied to clipboard

Yejin Bang, Nayeon Lee, Pascale Fung

Challenge:	polarity is a pervasive problem in modern media, misleading the understanding of what really happened via a skewed selection of information and language.
Approach:	They propose a loss function that encourages the model to minimize the polarity difference between the skewed input articles to reduce framing bias.
Outcome:	The proposed loss improves the model's ability to map polarity ends bidirectionally.

Zero-shot Cross-lingual Dialogue Systems with Transferable Latent Variables (D19-1)

Copied to clipboard

Zihan Liu, Jamin Shin, Yan Xu, Genta Indra Winata, Peng Xu, Andrea Madotto, Pascale Fung

Low-Resource NLP

Challenge:	a lack of research on multilingual or cross-lingual task-oriented dialog systems has limited results . we propose a zero-shot adaptation of task-orientated dialog systems to low-resource languages . task-focused systems are often trained with monolingual datasets that are expensive to build or acquire .
Approach:	They propose a zero-shot adaptation of multilingual task-oriented dialog systems to low-resource languages using latent variables and a set of very few parallel word pairs.
Outcome:	The proposed model performs better in natural language understanding task compared to state-of-the-art model . the proposed model uses very few parallel word pairs to refine cross-lingual representations .

IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation (2021.emnlp-main)

Copied to clipboard

Samuel Cahyawijaya, Genta Indra Winata, Bryan Wilie, Karissa Vincentio, Xiaohong Li, Adhiguna Kuncoro, Sebastian Ruder, Zhi Yuan Lim, Syafri Bahar, Masayu Khodra, Ayu Purwarianti, Pascale Fung

Language Models

Challenge:	Lack of publicly available NLG benchmarks for low-resource languages poses a challenge . authors show that IndoBART and IndoGPT achieve competitive performance on all tasks .
Approach:	They propose a benchmark to measure natural language generation progress in three low-resource languages of Indonesia . they use a corpus of pretraining datasets to build their models .
Outcome:	The proposed benchmark measures progress in Indonesian, Javanese, and Sundanese . the results highlight the importance of pretraining on closely related, localized languages .

Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization (2021.emnlp-main)

Copied to clipboard

Tiezheng Yu, Wenliang Dai, Zihan Liu, Pascale Fung

Generalizing Question Answering System with Pre-trained Language Model Fine-tuning (D19-58)

Challenge:	Existing MAS models cannot leverage GPLMs’ powerful generation ability.
Approach:	They propose a method to construct vision guided (VG) GPLMs that incorporate visual information while maintaining their original text generation ability.
Outcome:	The proposed model outperforms the previous state-of-the-art model by 5.7 ROUGE-1, 5.3 ROUGe-2, and 5.1 ROUGEL-L scores on the How2 dataset and contributes 83.6% of the overall improvement.

Copied to clipboard

Dan Su, Yan Xu, Genta Indra Winata, Peng Xu, Hyeondey Kim, Zihan Liu, Pascale Fung

Read before Generate! Faithful Long Form Question Answering with Machine Reading (2022.findings-acl)

Challenge:	Existing methods focus on improving in-domain performance, leaving open the question of how they can generalize to out-of-domain and unseen RC tasks.
Approach:	They propose a multi-task learning framework that learns the shared representation across different tasks and builds on a large pre-trained language model and fine-tuned on multiple RC datasets.
Outcome:	The proposed framework improves the BERT-Large baseline by 8.39 and 7.22 respectively.

Copied to clipboard

Dan Su, Xiaoguang Li, Jindi Zhang, Lifeng Shang, Xin Jiang, Qun Liu, Pascale Fung

Towards Mitigating LLM Hallucination via Self Reflection (2023.findings-emnlp)

Challenge:	Long-form question answering (LFQA) generates a paragraph-length answer for a given question.
Approach:	They propose a framework that jointly models answer generation and machine reading.
Outcome:	The proposed model generates a more factually accurate answer from millions of documents retrieved from a large dataset.

Copied to clipboard

Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, Pascale Fung

Reasoning

Challenge:	Large language models have shown promise for generative and knowledge-intensive tasks including question-answering (QA) but the practical deployment still faces challenges, notably the issue of “hallucination”, where models generate plausible-sounding but unfaithful or nonsensical information.
Approach:	They propose a self-reflection methodology that incorporates knowledge acquisition and answer generation to address the issue of "hallucination" they use a set of LLMs to generate a more accurate and factually accurate answer.
Outcome:	The proposed approach improves factuality, consistency, and entailment of the generated answers.

Generating Hashtags for Short-form Videos with Guided Signals (2023.acl-long)

Copied to clipboard

Tiezheng Yu, Hanchao Yu, Davis Liang, Yuning Mao, Shaoliang Nie, Po-Yao Huang, Madian Khabsa, Pascale Fung, Yi-Chia Wang

Information Retrieval

Challenge:	Short-form video hashtag recommendation (SVHR) is a classification or ranking problem that selects hashtags from a set of limited candidates.
Approach:	They propose a short-form video hashtag recommendation task that better represents how hashtags are created naturally by retrieving relevant hashtags from a large-scale hashtag pool as extra guidance signals.
Outcome:	The proposed model outperforms strong classification baselines on two short-form video datasets and the guidance signals boost the performance by 8.11 and 2.17 absolute ROUGE-1 scores on average.

Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data (2021.acl-long)

Copied to clipboard

Wei-Jen Ko, Ahmed El-Kishky, Adithya Renduchintala, Vishrav Chaudhary, Naman Goyal, Francisco Guzmán, Pascale Fung, Philipp Koehn, Mona Diab

Multilinguality

Challenge:	linguistic overlap between low-resource languages and high-resourced languages is a major obstacle for training high-quality machine translation systems.
Approach:	They exploit linguistic overlap to facilitate translation to and from low-resource languages . they use monolingual data and parallel data in related high-resourced languages based on their method .
Outcome:	The proposed method significantly improves translation into low-resource language compared to baselines on 7 languages from three different language families.

NusaCrowd: Open Source Initiative for Indonesian NLP Resources (2023.findings-acl)

Copied to clipboard

Samuel Cahyawijaya, Holy Lovenia, Alham Fikri Aji, Genta Winata, Bryan Wilie, Fajri Koto, Rahmad Mahendra, Christian Wibisono, Ade Romadhony, Karissa Vincentio, Jennifer Santoso, David Moeljadi, Cahya Wirawan, Frederikus Hudi, Muhammad Satrio Wicaksono, Ivan Parmonangan, Ika Alfina, Ilham Firdausi Putra, Samsul Rahmadani, Yulianti Oenang, Ali Septiandri, James Jaya, Kaustubh Dhole, Arie Suryani, Rifki Afina Putri, Dan Su, Keith Stevens, Made Nindyatama Nityasya, Muhammad Adilazuarda, Ryan Hadiwijaya, Ryandito Diandaru, Tiezheng Yu, Vito Ghifari, Wenliang Dai, Yan Xu, Dyah Damapuspita, Haryo Wibowo, Cuk Tho, Ichwanul Karo Karo, Tirana Fatyanosa, Ziwei Ji, Graham Neubig, Timothy Baldwin, Sebastian Ruder, Pascale Fung, Herry Sujaini, Sakriani Sakti, Ayu Purwarianti

Challenge:	Existing NLP research in Indonesian languages has been held back by factors such as language diversity, orthographic variation, resource limitation and other societal challenges.
Approach:	They present a collaborative initiative to collect and unify existing resources for Indonesian languages and open access to previously non-public resources.
Outcome:	The results show that the datasets are highly reliable and can be used to generate the first zero-shot benchmarks for natural language understanding and generation in Indonesian and the local languages of Indonesia.

Getting To Know You: User Attribute Extraction from Dialogues (2020.lrec-1)

Copied to clipboard

Chien-Sheng Wu, Andrea Madotto, Zhaojiang Lin, Peng Xu, Pascale Fung

Towards Few-shot Fact-Checking via Perplexity (2021.naacl-main)

Challenge:	a new method to extract user attributes from dialogues is needed to improve user understanding.
Approach:	They propose to leverage dialogues with conversational agents to automatically extract user attributes from dialogues.
Outcome:	The proposed model surpasses retrieval and generation baselines on human evaluation.

Copied to clipboard

Nayeon Lee, Yejin Bang, Andrea Madotto, Pascale Fung

Clickbait? Sensational Headline Generation with Auto-tuned Reinforcement Learning (D19-1)

Challenge:	Recent studies have shown that pre-trained language models can perform few-shot learning for various downstream tasks, such as question answering and machine translation.
Approach:	They propose a method to leverage the powerful transfer learning ability of a language model via a perplexity score to learn few-shot for the fact-checking task.
Outcome:	The proposed method outperforms the Major Class baseline by 10% on the F1-Macro metric across multiple datasets.

Copied to clipboard

Peng Xu, Chien-Sheng Wu, Andrea Madotto, Pascale Fung

Text Generation

Challenge:	Conventional abstractive headline generation methods do not optimize for maximum reader attention.
Approach:	They propose a model that generates sensational headlines without labeled data by classifying online headlines with many comments against a summarization model.
Outcome:	The proposed model generates sensational headlines without labeled data.

MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems (2020.emnlp-main)

Copied to clipboard

Zhaojiang Lin, Andrea Madotto, Genta Indra Winata, Pascale Fung

Improving Large-Scale Fact-Checking using Decomposable Attention Models and Lexical Tagging (D18-1)

Challenge:	Existing approaches to learn dialogue state tracking and response generation are time-intensive and not transferable between domains.
Approach:	They propose a transfer learning framework that allows efficient dialogue state tracking with a minimal generation length.
Outcome:	The proposed framework improves the inference efficiency and improves state-of-the-art results on multi-domain multi-tasking systems.

Copied to clipboard

Nayeon Lee, Chien-Sheng Wu, Pascale Fung

Tagging

Challenge:	Existing pipelines for fact-checking of textual sources are limited . fact- checking of text sources requires a large knowledge base to extract relevant information .
Approach:	They propose a neural ranker that dynamically selects sentences to improve evidence retrieval . they incorporate lexical tagging methods into the pipeline framework to simplify the tasks .
Outcome:	The proposed model outperforms the existing TF-IDF method on a large-scale fact extraction and verification dataset with speedup.

Multimodal End-to-End Sparse Model for Emotion Recognition (2021.naacl-main)

Copied to clipboard

Wenliang Dai, Samuel Cahyawijaya, Zihan Liu, Pascale Fung

Sentiment Analysis

Challenge:	Existing work in emotion recognition uses a two-phase pipeline, but the extracted features are fixed and cannot be fine-tuned on different tasks.
Approach:	They propose a two-phase pipeline for emotion recognition and personality recognition . they propose restructured datasets to enable fully end-to-end training .
Outcome:	The proposed model outperforms the current state-of-the-art models on emotion recognition and personality recognition tasks with half less computation in the feature extraction part.

A Submodular Feature-Aware Framework for Label Subset Selection in Extreme Classification Problems (N19-1)

Copied to clipboard

Elham J. Barezi, Ian D. Wood, Pascale Fung, Hamid R. Rabiee

Information Extraction & Text Mining

Challenge:	Experimental results show that extreme multi-label learning improves label prediction quality by 3% to 5% in three of the 5 tasks and is competitive in the others.
Approach:	They propose a submodular maximization framework with linear cost to find informative labels which are most relevant to other labels yet least redundant with each other.
Outcome:	The proposed model improves label prediction quality by 3% to 5% in three of the 5 tasks and is competitive in the others.

QAConv: Question Answering on Informative Conversations (2022.acl-long)

Copied to clipboard

Chien-Sheng Wu, Andrea Madotto, Wenhao Liu, Pascale Fung, Caiming Xiong

Coach: A Coarse-to-Fine Approach for Cross-domain Slot Filling (2020.acl-main)

Challenge:	Experimental results show that state-of-the-art pretrained QA systems have limited zero-shot performance and tend to predict our questions as unanswerable.
Approach:	They propose a question-answering dataset that uses conversations as a knowledge source.
Outcome:	The proposed dataset provides a training and evaluation testbed to facilitate QA on conversations research.

Copied to clipboard

Zihan Liu, Genta Indra Winata, Peng Xu, Pascale Fung

Semantic Parsing

Challenge:	Existing approaches to slot filling are expensive and time-consuming.
Approach:	They propose a Coarse-to-fine approach for cross-domain slot filling . they propose utterance templates to regularize the representation of utterrances .
Outcome:	The proposed model outperforms state-of-the-art approaches in slot filling . it can be applied to the cross-domain named entity recognition task .

Investigating Audio, Video, and Text Fusion Methods for End-to-End Automatic Personality Prediction (P18-2)

Copied to clipboard

Onno Kampman, Elham J. Barezi, Dario Bertero, Pascale Fung

Improve Query Focused Abstractive Summarization by Incorporating Answer Relevance (2021.findings-acl)

Challenge:	Using stacked Convolutional Neural Networks, we can predict personality traits from video clips with different channels for audio, text, and video data.
Approach:	They propose a tri-modal architecture to predict Big Five personality trait scores from video clips with different channels for audio, text, and video data.
Outcome:	The proposed model outperforms the best individual modality with 9.4% accuracy over the best channel.

Copied to clipboard

Dan Su, Tiezheng Yu, Pascale Fung

Summarization

Challenge:	Query focused summarization models aim to generate summaries from source documents that can answer the given query.
Approach:	They propose a QFS-BART model that incorporates the explicit answer relevance of the source documents given the query via a question answering model.
Outcome:	Empirical results show that the proposed model achieves the new state-of-the-art performance.

Measuring Political Bias in Large Language Models: What Is Said and How It Is Said (2024.acl-long)

Copied to clipboard

Yejin Bang, Delong Chen, Nayeon Lee, Pascale Fung

LLMs Are Few-Shot In-Context Low-Resource Language Learners (2024.naacl-long)

Challenge:	Existing benchmarks and measures focus on gender and racial biases, but political bias exists in LLMs and can lead to polarization and other harms in downstream applications.
Approach:	They propose to analyze the content and style of LLMs generated by political issues and propose a framework that can be scalable to other topics.
Outcome:	The proposed framework is easily scalable to other topics and is explainable.

Copied to clipboard

Samuel Cahyawijaya, Holy Lovenia, Pascale Fung

Low-Resource NLP

Challenge:	In-context learning (ICL) empowers large language models to perform diverse tasks in underrepresented languages using only short in-contrast information.
Approach:	They extensively assess the effectiveness of in-context learning with LLMs in low-resource languages . they also identify the shortcomings of in context label alignment .
Outcome:	The proposed approach improves understanding quality of low-resource languages by closing the language gap in the target language.

IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding (2020.aacl-main)

Copied to clipboard

Bryan Wilie, Karissa Vincentio, Genta Indra Winata, Samuel Cahyawijaya, Xiaohong Li, Zhi Yuan Lim, Sidik Soleman, Rahmad Mahendra, Pascale Fung, Syafri Bahar, Ayu Purwarianti

Challenge:	Despite the availability of data on Indonesian, progress on this language is slow . available datasets are scattered, with a lack of documentation and minimal community engagement.
Approach:	They propose a resource for training, evaluation, and benchmarking on Indonesian natural language understanding tasks.
Outcome:	The proposed resource includes 12 tasks ranging from single sentence classification to pair-sentences sequence labeling with different levels of complexity.

Belief Revision: The Adaptability of Large Language Models Reasoning (2024.emnlp-main)

Copied to clipboard

Bryan Wilie, Samuel Cahyawijaya, Etsuko Ishii, Junxian He, Pascale Fung

Language Models

Challenge:	Existing evaluations assume language models operate with consistent information.
Approach:	They propose a dataset to test LMs' belief revision ability when presented with new evidence.
Outcome:	The proposed framework improves language models’ adaptiveness to changing information, highlighting a critical trade-off.

Hierarchical Meta-Embeddings for Code-Switching Named Entity Recognition (D19-1)

Copied to clipboard

Genta Indra Winata, Zhaojiang Lin, Jamin Shin, Zihan Liu, Pascale Fung

Representation Learning

Challenge:	Existing work on name-switching focuses on word-level aspects but neglects subword-level characteristics shared across languages.
Approach:	They propose hierarchical meta-Embeddings that combine word-level and subword-level embeddings to create language-agnostic lexical representations.
Outcome:	The proposed model achieves state-of-the-art in English-Spanish code-switching scenarios.

CI-AVSR: A Cantonese Audio-Visual Speech Datasetfor In-car Command Recognition (2022.lrec-1)

Copied to clipboard

Wenliang Dai, Samuel Cahyawijaya, Tiezheng Yu, Elham J. Barezi, Peng Xu, Cheuk Tung Yiu, Rita Frieske, Holy Lovenia, Genta Winata, Qifeng Chen, Xiaojuan Ma, Bertram Shi, Pascale Fung

Meta-Transfer Learning for Code-Switched Speech Recognition (2020.acl-main)

Challenge:	In-car smart assistants should be able to process general as well as car-related commands and perform corresponding actions, which eases driving and improves safety.
Approach:	They propose a dataset for in-car command recognition in the cantonese language with both video and audio data.
Outcome:	The proposed model can achieve a considerable quality on the clean test set, but the speech recognition quality on noisy data is still inferior.

Copied to clipboard

Genta Indra Winata, Samuel Cahyawijaya, Zhaojiang Lin, Zihan Liu, Peng Xu, Pascale Fung

ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversation (2022.lrec-1)

Challenge:	Increasing number of people in the world today speak a mixed-language as a result of being multilingual.
Approach:	They propose a method to transfer learn on a code-switched speech recognition system by extracting information from high-resource monolingual datasets.
Outcome:	The proposed model outperforms baselines on speech recognition and language modeling tasks and is faster to converge.

Copied to clipboard

Holy Lovenia, Samuel Cahyawijaya, Genta Winata, Peng Xu, Yan Xu, Zihan Liu, Rita Frieske, Tiezheng Yu, Wenliang Dai, Elham J. Barezi, Qifeng Chen, Xiaojuan Ma, Bertram Shi, Pascale Fung

High-Dimension Human Value Representation in Large Language Models (2025.naacl-long)

Challenge:	Code-switching is a speech phenomenon occurring when a speaker switches language during a conversation.
Approach:	They propose to collect Mandarin Chinese-English code-switching corpus from read speech rather than spontaneous speech to address this phenomenon.
Outcome:	ASCEND consists of 10.62 hours of clean speech, collected from 23 bilingual speakers of Chinese and English.

Copied to clipboard

Samuel Cahyawijaya, Delong Chen, Yejin Bang, Leila Khalatbari, Bryan Wilie, Ziwei Ji, Etsuko Ishii, Pascale Fung

Which One Are You Referring To? Multimodal Object Identification in Situated Dialogue (2023.eacl-srw)

Challenge:	Existing approaches to align large language models with human values and preferences are not able to be applied to all tasks and fields.
Approach:	They propose a high-dimensional representation of symbolic human value distributions in LLMs that is orthogonal to model architecture and training data.
Outcome:	The proposed representations are evaluated on 15 open-source and commercial LLMs and are self-supervised from the value-relevant output of 8 LLM models.

Copied to clipboard

Holy Lovenia, Samuel Cahyawijaya, Pascale Fung

HalluLens: LLM Hallucination Benchmark (2025.acl-long)

Challenge:	a demand for multimodal dialogue systems is increasing for situated dialogues, where a dialogue agent shares a co-observed vision or physical space with the user.
Approach:	They propose three methods to solve multimodal object identification problem using situated dialogue dataset SIMMC 2.1.
Outcome:	The proposed method improves by 20% F1-score on the largest situated dialogue dataset, SIMMC 2.1.

Copied to clipboard

Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda, Pascale Fung

Reasoning

Challenge:	Large language models (LLMs) generate responses that deviate from user input or training data, a phenomenon known as "hallucination" .
Approach:	They propose a hallucination benchmark HalluLens that includes both extrinsic and intrinsic evaluation tasks to distinguish between extrindic and intrinsic hallucines.
Outcome:	The proposed framework disentangles LLM hallucination from "factuality" and distinguishes between extrinsic and intrinsic hallucines to promote consistency and facilitate research.

Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems (P19-1)

Copied to clipboard

Chien-Sheng Wu, Andrea Madotto, Ehsan Hosseini-Asl, Caiming Xiong, Richard Socher, Pascale Fung

Plug-and-Play Conversational Models (2020.findings-emnlp)

Challenge:	Existing approaches to dialogue state tracking are dependent on domain ontology and lack of sharing knowledge across domains.
Approach:	They propose a transferable dialogue state generator that generates dialogue states from utterances using copy mechanism.
Outcome:	Empirical results show that TRADE achieves state-of-the-art 48.62% joint goal accuracy for the five domains of MultiWOZ.

Copied to clipboard

Andrea Madotto, Etsuko Ishii, Zhaojiang Lin, Sumanth Dathathri, Pascale Fung

Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation (2022.findings-acl)

Challenge:	Large conversational models that generate coherent and fluent responses often require large dialogue datasets.
Approach:	They propose and evaluate plug-and-play methods for controllable response generation . they demonstrate a high degree of control over the generated conversational responses .
Outcome:	The proposed method does not require further computation at decoding time and does not need fine-tuning of a large language model.

Copied to clipboard

Wenliang Dai, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu, Pascale Fung

NeuS: Neutral Multi-News Summarization for Mitigating Framing Bias (2022.naacl-main)

Challenge:	Recent large-scale vision-language pre-training models are powerful in multimodal classification and retrieval tasks.
Approach:	They propose to augment a vision-language pre-training model with a textual pre-trained language model . the model achieves 44.5% zero-shot accuracy on multimodal generation tasks .
Outcome:	The proposed model achieves 44.5% zero-shot accuracy on open-ended visual question answering and image captioning tasks.

Copied to clipboard

Nayeon Lee, Yejin Bang, Tiezheng Yu, Andrea Madotto, Pascale Fung

Summarization

Challenge:	a new task is proposed to reduce media news framing bias by generating a neutral summary from multiple news articles of the varying political leanings.
Approach:	They propose a task that generates a neutral summary from multiple news articles . they find title provides a good signal for framing bias and propose metric and model .
Outcome:	The proposed task can neutralize news content in hierarchical order from title to article . scalability remains a bottleneck due to the time-consuming human labor needed for composing the roundup .

Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning (2020.findings-emnlp)

Copied to clipboard

Zhaojiang Lin, Andrea Madotto, Pascale Fung

Language Models

Challenge:	Large-scale language models can be fine-tuned to learn highly transferable embedding, but they are expensive and require multiple model parameters.
Approach:	They propose a way to fine-tune multiple down-stream generation tasks simultaneously using a single, large pretrained model.
Outcome:	The proposed model can maintain or improve the performance of fine-tuning the whole model.

RHO: Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding (2023.findings-acl)

Copied to clipboard

Ziwei Ji, Zihan Liu, Nayeon Lee, Tiezheng Yu, Bryan Wilie, Min Zeng, Pascale Fung

Structured Data in NLP

Challenge:	Existing knowledge-grounded dialogue systems generate accurate and informative responses, but they are prone to hallucination problems.
Approach:	They propose a method to generate hallucinated responses using knowledge graphs . they propose local knowledge grounding to combine textual embeddings with corresponding KG embeddments . a global knowledge ground technique is also proposed to equip RHO with multi-hop reasoning abilities .
Outcome:	The proposed approach outperforms state-of-the-art methods on automatic and human evaluation by a large margin.

Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training (2023.eacl-main)

Copied to clipboard

Wenliang Dai, Zihan Liu, Ziwei Ji, Dan Su, Pascale Fung

Continual Mixed-Language Pre-Training for Extremely Low-Resource Neural Machine Translation (2021.findings-acl)

Challenge:	Large-scale vision-language pre-trained (VLP) models generate unfaithful or nonsensical texts given the source input, which is called hallucination.
Approach:	They propose a VLP loss-based model to mitigate object hallucination by decoupling VLP objectives and a token-level image-text alignment.
Outcome:	The proposed model reduces object hallucination by 17.4% on two benchmarks.

Copied to clipboard

Zihan Liu, Genta Indra Winata, Pascale Fung

Multilinguality

Challenge:	a lack of data in low-resource languages has limited the performance of a multilingual pre-trained model.
Approach:	They propose a continuous pre-training framework to adapt mBART to unseen languages . they construct noisy mixed-language text from the monolingual corpus of the target language .
Outcome:	The proposed framework improves finetuning performance on low-resource translation pairs . the proposed framework also improves on translation pairs where both languages are seen .

Continual Learning in Task-Oriented Dialogue Systems (2021.emnlp-main)

Copied to clipboard

Andrea Madotto, Zhaojiang Lin, Zhenpeng Zhou, Seungwhan Moon, Paul Crook, Bing Liu, Zhou Yu, Eunjoon Cho, Pascale Fung, Zhiguang Wang

Modality-Transferable Emotion Embeddings for Low-Resource Multimodal Emotion Recognition (2020.aacl-main)

Challenge:	Existing continuous learning systems are not designed to add new domains and functionalities through time without incurring the high cost of retraining the whole system.
Approach:	They propose a first-ever continual learning benchmark for task-oriented dialogue systems . they propose 'architecture' method based on residual adapters to implement continual training .
Outcome:	The proposed architectural method performs better than multitask learning while being 20X faster in learning new domains.

Copied to clipboard

Wenliang Dai, Zihan Liu, Tiezheng Yu, Pascale Fung

Cross-lingual Spoken Language Understanding with Regularized Representation Alignment (2020.emnlp-main)

Challenge:	despite recent advances in multimodal emotion recognition, two problems still exist: sub-optimal performance and low-resource emotions.
Approach:	They propose a modality-transferable model with emotion embeddings to solve these problems . they use pre-trained word embedders to represent emotion categories for textual data .
Outcome:	The proposed model outperforms baselines in zero-shot and few-shot scenarios for unseen emotions.

Copied to clipboard

Zihan Liu, Genta Indra Winata, Peng Xu, Zhaojiang Lin, Pascale Fung

Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations (2025.emnlp-main)

Challenge:	despite promising results, current cross-lingual models suffer from imperfect cross-linguistic representation alignments between the source and target languages, which makes the performance sub-optimal.
Approach:	They propose a regularization approach to align word-level and sentence-level representations across languages without external resources.
Outcome:	The proposed model outperforms state-of-the-art models in few-shot and zero-shot scenarios and achieves comparable performance to supervised training with all training data.

Copied to clipboard

Ziwei Ji, Lei Yu, Yeskendir Koishekenov, Yejin Bang, Anthony Hartshorn, Alan Schelten, Cheng Zhang, Pascale Fung, Nicola Cancedda

Reasoning

Challenge:	LLMs often use assertive language when making false claims, resulting in harm and loss of trust.
Approach:	They find that a mismatch between semantic and verbal uncertainty is a better predictor of hallucinations than semantic uncertainty alone.
Outcome:	a new study shows that mismatch between semantic and verbal uncertainty is better predictor of hallucinations than semantic uncertainty alone.

RoAST: Robustifying Language Models via Adversarial Perturbation with Selective Training (2023.findings-emnlp)

Copied to clipboard

Jaehyung Kim, Yuning Mao, Rui Hou, Hanchao Yu, Davis Liang, Pascale Fung, Qifan Wang, Fuli Feng, Lifu Huang, Madian Khabsa

Robustness in NLP

Challenge:	Several perspectives of robustness for pre-trained language models have been studied independently, but lacking a unified consideration in multiple perspectives.
Approach:	They propose a technique to enhance the multi-perspective robustness of LMs by introducing adversarial perturbation while the model parameters are selectively updated upon their relative importance.
Outcome:	The proposed technique improves the robustness of LMs by incorporating four perspectives on model robustness.

Contrastive Learning for Inference in Dialogue (2023.emnlp-main)

Copied to clipboard

Etsuko Ishii, Yan Xu, Bryan Wilie, Ziwei Ji, Holy Lovenia, Willy Chung, Pascale Fung

On Unifying Misinformation Detection (2021.naacl-main)

Challenge:	Recent large language models show remarkable advances in inference tasks, but their performance in inductive reasoning is far behind deductive reasoning.
Approach:	They propose to use negative samples to analyze inferences based on the semantic information gap between dialogue contexts and desired inference.
Outcome:	The proposed model improves inference generation by feeding negative samples to the models.

Copied to clipboard

Nayeon Lee, Belinda Z. Li, Sinong Wang, Pascale Fung, Hao Ma, Wen-tau Yih, Madian Khabsa

Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems (2020.findings-emnlp)

Challenge:	On any given day, 2.5 quintillion bytes of information are created on the Internet, a figure that is only expected to increase in the coming years.
Approach:	They propose a general-purpose misinformation model that jointly models multiple domains of misinformation with a single, unified setup.
Outcome:	The proposed model is useful for few-shot learning of unseen misinformation tasks/datasets and generalizability to unseense events.

Copied to clipboard

Andrea Madotto, Samuel Cahyawijaya, Genta Indra Winata, Yan Xu, Zihan Liu, Zhaojiang Lin, Pascale Fung

ToKen: Task Decomposition and Knowledge Infusion for Few-Shot Hate Speech Detection (2022.emnlp-main)

Challenge:	End-to-end systems rely on dialogue state tracking and annotations to fulfill user requests . modularized systems require multiple steps, including a direct interaction with the KB .
Approach:	They propose a method to embed the KB directly into the model parameters . they evaluate five task-oriented dialogue datasets with small, medium, and large KBs .
Outcome:	The proposed model can embed the KB directly into the model parameters without any DST or template responses, nor the kb as input.

Copied to clipboard

Badr AlKhamissi, Faisal Ladhak, Srinivasan Iyer, Veselin Stoyanov, Zornitsa Kozareva, Xian Li, Pascale Fung, Lambert Mathias, Asli Celikyilmaz, Mona Diab

Integrating Question Rewrites in Conversational Question Answering: A Reinforcement Learning Approach (2022.acl-srw)

Challenge:	Hate speech detection is complex and requires commonsense reasoning and social nuance . prior work has shown that even humans cannot achieve a high agreement on whether a post constitutes HS .
Approach:	They frame a few-shot learning task to decompose a hate speech detection task into its "constituent" parts. they show that infusing commonsense knowledge from reasoning datasets improves the performance even further.
Outcome:	The proposed method outperforms baseline methods in the 16-shot case.

Copied to clipboard

Etsuko Ishii, Bryan Wilie, Yan Xu, Samuel Cahyawijaya, Pascale Fung