Papers by Yaobo Liang
Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks (D19-1)
Copied to clipboard
| Challenge: | Existing models that can handle cross-lingual tasks with limited or no training data are insensitive to different languages. |
| Approach: | They propose to use Unicoder to train models in one language and apply it to other languages. |
| Outcome: | Experiments show that Unicoder learns the mappings among different languages from more perspectives. |
Large Language Models Can Learn Representation in Natural Language (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are unable to complete complex tasks involving multiple entities, such as tool APIs. |
| Approach: | They propose a method which uses natural language representations to refine entity descriptions for improved retrieval and LLM utilization. |
| Outcome: | The proposed method improves GPT-4's performance on classification tasks and API call tasks. |
Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to improve machine reading comprehension performance on low resource languages are limited due to the lack of sufficient training data. |
| Approach: | They propose to use a mixed MRC task to translate the question to other languages and build cross-lingual question-passage pairs. |
| Outcome: | The proposed task improves on two cross-lingual MRC datasets. |
Learning to Plan by Updating Natural Language (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown remarkable performance in basic natural language tasks. |
| Approach: | They propose a method that iteratively updates the task plan with new steps and behavioral instructions to guide LLMs to generate the correct solutions step by step. |
| Outcome: | The proposed method improves performance on five reasoning type tasks and can be directly applied to other LLMs. |
Analyzing and Reducing the Performance Gap in Cross-Lingual Transfer with Fine-tuning Slow and Fast (2023.acl-long)
Copied to clipboard
| Challenge: | Existing research shows that a multilingual pre-trained language model fine-tuned with one (source) language performs well on downstream tasks for non-source languages . However, there is a clear performance gap between the source and non-sourced languages - this gap can be reduced by reducing forgetting. |
| Approach: | They propose a method to fine-tune a multilingual pre-trained language model fine- tuned with one (source) language and four training policies to address the performance gap. |
| Outcome: | The proposed method outperforms baselines on the XNLI dataset by a clear margin. |
PPTC-R benchmark: Towards Evaluating the Robustness of Large Language Models for PowerPoint Task Completion (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used for task completion in real-world situations. |
| Approach: | They propose a PowerPoint Task Completion-Robustness (PPTC-R) benchmark to measure LLMs’ robustness to the user PPT task instruction and software version (Powerpoint). |
| Outcome: | The proposed benchmark compares 3 closed-source and 4 open-source LLMs to the PowerPoint task instruction and software version (Powerpoint) . |
Low-code LLM: Graphical User Interface over Large Language Models (2024.naacl-demo)
Copied to clipboard
Yuzhe Cai, Shaoguang Mao, Wenshan Wu, Zehua Wang, Yaobo Liang, Tao Ge, Chenfei Wu, WangYou WangYou, Ting Song, Yan Xia, Nan Duan, Furu Wei
| Challenge: | Low-code LLM is a visual programming interface that allows users to incorporate their ideas into the process without writing trivial prompts. |
| Approach: | They propose a human-LLM interaction framework that incorporates low-code visual programming interactions to achieve more controllable and stable responses. |
| Outcome: | The proposed framework enables users to incorporate ideas into the process without writing trivial prompts. |
Multi-View Document Representation Learning for Open-Domain Dense Retrieval (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for dense retrieval are hard to match with multiple views. |
| Approach: | They propose a multi-view document representation learning framework to generate multiple embeddings through viewers to represent documents and enforce them to align with different queries. |
| Outcome: | The proposed method outperforms recent works and achieves state-of-the-art results. |
Cross-Lingual Ability of Multilingual Masked Language Models: A Study of Language Structure (2022.acl-long)
Copied to clipboard
| Challenge: | Multilingual pre-trained language models have shown impressive cross-lingual ability. |
| Approach: | They argue that cross-language ability comes from commonality between languages . they create an artificial language by modifying property in source language . |
| Outcome: | The proposed model can be implemented in multilingual and low-resource language scenarios without cross-lingual supervision or aligned data. |
PPTC Benchmark: Evaluating Large Language Models for PowerPoint Task Completion (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent evaluations of Large Language Models (LLMs) focus on their zero-shot/few-shot capabilities for basic natural language tasks and their ability to translate instructions into tool APIs. |
| Approach: | They propose a PowerPoint Task Completion benchmark to assess LLMs’ ability to create and edit PPT files based on user instructions. |
| Outcome: | The proposed system outperforms open-source and closed LLMs with 75.1% accuracy in single-turn dialogue testing but only achieves 6% session accuracy. |
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation (2020.emnlp-main)
Copied to clipboard
Yaobo Liang, Nan Duan, Yeyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, Xiaodong Fan, Ruofei Zhang, Rahul Agrawal, Edward Cui, Sining Wei, Taroon Bharti, Ying Qiao, Jiun-Hung Chen, Winnie Wu, Shuguang Liu, Fan Yang, Daniel Campos, Rangan Majumder, Ming Zhou
| Challenge: | XGLUE provides a benchmark dataset to train large-scale cross-lingual pre-trained models . XCLUE provides 11 diversified tasks that cover both understanding and generation scenarios . |
| Approach: | They introduce a new benchmark dataset to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora. |
| Outcome: | The proposed dataset is labeled in English and includes only natural language understanding tasks. |
Dense Procedure Captioning in Narrated Instructional Videos (P19-1)
Copied to clipboard
| Challenge: | Existing models for video dense captioning learn video segments and generate captions without considering transcripts. |
| Approach: | They propose a model to generate procedure captions from narrated instructional videos . they extract procedures by a cross-modality module and generate captions by encoding video frames and transcripts within each extracted procedure. |
| Outcome: | The proposed model can extract procedures from narrated instructional videos and generate procedure captions by encoding video frames and transcripts. |
AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models (2024.findings-naacl)
Copied to clipboard
Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, Nan Duan
| Challenge: | Traditional benchmarks for evaluating foundation models often fail to accurately represent their general abilities for human-centric tasks. |
| Approach: | They propose a bilingual benchmark to assess foundation models in the context of human-centric standardized exams such as college entrance exams, law school admission tests, and math competitions. |
| Outcome: | The proposed benchmark exceeds the average human performance on SAT, LSAT, and math competitions with 95% accuracy and 92.5% on the Chinese college entrance English exam. |
Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to machine reading comprehension treat documents at their hierarchical nature, ignoring their dependencies. |
| Approach: | They propose a machine reading comprehension benchmark with two-grained answers . they use graph attention networks to model documents at their hierarchical nature . |
| Outcome: | The proposed framework outperforms existing systems at long and short answer criteria. |
Competition-Level Problems are Effective LLM Evaluators (2024.findings-acl)
Copied to clipboard
Yiming Huang, Zhenghao Lin, Xiao Liu, Yeyun Gong, Shuai Lu, Fangyu Lei, Yaobo Liang, Yelong Shen, Chen Lin, Nan Duan, Weizhu Chen
| Challenge: | Large language models (LLMs) have demonstrated impressive reasoning capabilities, yet there is ongoing debate about their capabilities and the potential data contamination problem. |
| Approach: | They propose to evaluate the reasoning capabilities of large language models in solving recent competition-level programming problems in Codeforces. |
| Outcome: | The proposed model has experienced a cliff-like decline in problems after September 2021, which shows the potential data contamination and the challenges for any existing LLM to solve unseen complex reasoning problems. |
Discovering Representation Sprachbund For Multilingual Pre-Training (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models perform poorly on many languages and cross-lingual tasks due to typological differences and contradictions between some languages. |
| Approach: | They propose to pre-train multilingual pre-trained models to handle cross-lingual tasks in one model. |
| Outcome: | The proposed model improves performance on cross-lingual tasks compared to baselines on multiple languages . |