Papers by Xuedong Huang
Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization (2023.acl-long)
Copied to clipboard
Pengcheng He, Baolin Peng, Song Wang, Yang Liu, Ruochen Xu, Hany Hassan, Yu Shi, Chenguang Zhu, Wayne Xiong, Michael Zeng, Jianfeng Gao, Xuedong Huang
| Challenge: | Z-Code++ is a pre-trained language model optimized for abstractive text summarization. |
| Approach: | They propose a pre-trained language model optimized for abstractive text summarization that uses a two-phase pre-training technique to improve model's performance. |
| Outcome: | The proposed model outperforms the competing models on low-resource summarization tasks in zero-shot and few-shot settings. |
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data (2024.findings-naacl)
Copied to clipboard
Ziyi Yang, Mahmoud Khademi, Yichong Xu, Reid Pryzant, Yuwei Fang, Chenguang Zhu, Dongdong Chen, Yao Qian, Xuemei Gao, Yi-Ling Chen, Robert Gmyr, Naoyuki Kanda, Noel Codella, Bin Xiao, Yu Shi, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang
| Challenge: | i-Code V2 is one of the first models capable of generating natural language from any combination of Vision, Language, and Speech data. |
| Approach: | They propose to create a model that can generate natural language from any combination of Vision, Language, and Speech data. |
| Outcome: | i-Code V2 matches or outperforms state-of-the-art single- and dual-modality baselines on 7 multimodal tasks. |
Fusing Context Into Knowledge Graph for Commonsense Question Answering (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to combine language modeling and knowledge graphs (KG) lack the context to provide a more precise understanding of the concepts. |
| Approach: | They propose to use external entity descriptions to provide contextual information for commonsense question answering models. |
| Outcome: | The proposed model achieves state-of-the-art among non-generative models in OpenBookQA and is the first of its kind in the field. |
A Hierarchical Network for Abstractive Meeting Summarization with Cross-Domain Pretraining (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods of summarizing meetings require complex multi-step pipelines that are intractable. |
| Approach: | They propose an abstractive summary network that adapts to meeting transcripts by hierarchical structure and role vectors. |
| Outcome: | The proposed model outperforms existing methods in both metrics and human evaluation. |
Mixed-Lingual Pre-training for Cross-lingual Summarization (2020.aacl-main)
Copied to clipboard
| Challenge: | Cross-lingual summarization (CLS) aims at producing a summary in the target language for an article in the source language. |
| Approach: | They propose a mixed-lingual pre-training scheme that leverages both cross-lingual tasks such as translation and monolingual tasks like masked language models. |
| Outcome: | The proposed model improves on the translation and masked language models with no task-specific components and saves memory. |
Multi-task Learning for Natural Language Generation in Task-Oriented Dialogue (D19-1)
Copied to clipboard
| Challenge: | Existing methods to generate natural language for task-oriented dialogues lack naturalness and variation in language. |
| Approach: | They propose a multi-task learning framework for natural language generation that explicitly targets for naturalness in generated responses via an unconditioned language model. |
| Outcome: | The proposed framework outperforms existing models across multiple datasets in the study of natural language generation. |
Enhancing Factual Consistency of Abstractive Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Abstractive summarization models often distort or fabricate facts in articles . factual inconsistency is a common problem with abstractive summaries . |
| Approach: | They propose a fact-aware summarization model FASum to extract factual relations into the summary generation process via graph attention. |
| Outcome: | The proposed model can produce abstractive summaries with higher factual consistency compared with existing systems and corrects factual errors via modifying only a few keywords. |
TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing abstractive summarization models ignore abundant unlabeled corpora resources . TED outperforms all unsupervised abstractive baselines on NYT, CNN/DM and English Gigaword datasets . |
| Approach: | They propose a transformer-based unsupervised text summarization system with pretraining on large-scale data. |
| Outcome: | The proposed system outperforms baseline models on NYT, CNN/DM and English Gigaword datasets with various document styles. |
i-Code Studio: A Configurable and Composable Framework for Integrative AI (2024.emnlp-demo)
Copied to clipboard
Yuwei Fang, Mahmoud Khademi, Chenguang Zhu, Ziyi Yang, Reid Pryzant, Yichong Xu, Yao Qian, Takuya Yoshioka, Lu Yuan, Michael Zeng, Xuedong Huang
| Challenge: | Existing frameworks for Integrative AI lack flexibility and composability to handle multimodal tasks. |
| Approach: | They propose a configurable framework for Integrative AI that orchestrates multiple pre-trained models to conduct complex multimodal tasks. |
| Outcome: | The proposed framework achieves impressive results on zero-shot multimodal tasks . it can communicate and personalize for users, and it can be used in a multimodal agent . |