Papers by Luo Ao
Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-training (2023.acl-long)
Copied to clipboard
| Challenge: | Empirical results show that CCLM significantly outperforms the prior state-of-the-art with an average absolute improvement of over 10%. |
| Approach: | They introduce a pre-training framework that unifies cross-lingual and cross-modal pre-trained models with shared architectures and objectives. |
| Outcome: | The proposed framework outperforms the state-of-the-art in two multi-lingual datasets and two multilingual image-text retrieval datasets. |
Better Explain Transformers by Illuminating Important Information (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing explanations focus on the input and output of the Transformers, resulting in confusing results. |
| Approach: | They propose to highlight important information and eliminate irrelevant information by a refined information flow on top of the layer-wise relevance propagation method. |
| Outcome: | The proposed method outperforms baseline models on classification and question-answering datasets with over 3% to 33% improvement on explanation metrics. |
Self Question-answering: Aspect-based Sentiment Analysis by Role Flipped Machine Reading Comprehension (2021.findings-emnlp)
Copied to clipboard
| Challenge: | unified Aspect-based Sentiment Analysis (ABSA) aims to couple aspect terms with their corresponding opinion terms, which might make it easier to predict sentiment polarities. |
| Approach: | They propose a new paradigm to pair aspect terms with their corresponding opinion terms . they propose to use a machine learning paradigm to solve the unified ABSA task . |
| Outcome: | The proposed framework can solve the ABSA task without any additional data annotation or transformation. |
Meet Changes with Constancy: Learning Invariance in Multi-Source Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to multi-source neural machine translation neglect inconsistencies between sources of information. |
| Approach: | They propose a source invariance network to learn invariant information of parallel sources . they propose to integrate such network with multi-encoder based multi-source NMT methods . |
| Outcome: | The proposed approach achieves clear gains in translation quality and captures implicit invariance between different sources. |
Reading Like HER: Human Reading Inspired Extractive Summarization (D19-1)
Copied to clipboard
| Challenge: | Existing methods for extracting text summarization are abstractive and extractive. |
| Approach: | They propose a novel approach for extractive summarization by simulating two stages . they adopt a convolutional neural network to encode gist of paragraphs for rough reading . |
| Outcome: | The proposed method significantly outperforms the state-of-the-art extractive methods on CNN and DailyMail datasets. |
Making Flexible Use of Subtasks: A Multiplex Interaction Network for Unified Aspect-based Sentiment Analysis (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies aim to integrate multiple sub-tasks into a unified ABSA model but suffer from major disadvantages . |
| Approach: | They propose a multi-task learning approach to make use of sub-tasks for a unified ABSA. |
| Outcome: | The proposed model can work well when some sub-tasks are absent, and the interactive relations among subtasks not adequate. |
Naive Bayes-based Context Extension for Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown promising in-context learning abilities, but are often impeded by length limitations of transformer architecture. |
| Approach: | They propose a framework to enable existing Large Language Models to perform ICL with an increased number of demonstration examples by significantly expanding their context size. |
| Outcome: | The proposed framework outperforms existing methods in in-context learning with an increased number of demonstration examples while preserving linear efficiency. |
WebSRC: A Dataset for Web-Based Structural Reading Comprehension (2021.emnlp-main)
Copied to clipboard
| Challenge: | Using a web page and a question, a machine can't understand the contents of web pages. |
| Approach: | They propose a novel dataset for web-based structural reading comprehension that consists of 400K question-answer pairs and a dataset of 6.4K web pages. |
| Outcome: | The proposed dataset consists of 400K question-answer pairs, collected from 6.4K web pages with corresponding HTML source code, screenshots, and metadata. |
PENS: A Dataset and Generic Framework for Personalized News Headline Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Using a dataset of Microsoft News, we propose a generic framework to personalize a text generator and establish personalized headlines. |
| Approach: | They propose a generic framework to personalize a news headline generator and establish personalized headlines by leveraging user behavioral data. |
| Outcome: | The proposed framework is based on user preference data and user preference injections to personalize a text generator and establish personalized headlines. |