Papers by Yuliang Cai

GenGO

Omakase Read Later

Search

Papers by Yuliang Cai

Findings

2023.findings-emnlp

transformer-based CL architecture
knowledge distillation
computational load
fine-tuning large-scale pre-trained neural network
machine learning
unimodal vision-only or language-only tasks
learning bimodal vision-and-language tasks
catastrophic forgetting
scalable learning
vision-and-language tasks

2023

Language Models

1 papers

Task-Attentive Transformer Architecture for Continual Learning of Vision-and-Language Tasks Using Knowledge Distillation (2023.findings-emnlp)

Copied to clipboard

Yuliang Cai, Jesse Thomason, Mohammad Rostami

Language Models

Challenge:	Existing algorithms for learning unimodal vision-only or language-only tasks are limited by the size and computational load of fine-tuning large-scale pre-trained neural networks.
Approach:	They propose a transformer-based CL architecture for learning bimodal vision-and-language tasks by increasing the number of the learnable parameters dynamically and using knowledge distillation.
Outcome:	The proposed model reaches state-of-the-art on vision-and-language tasks.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations