Papers by Roy Lee
Language Guided Visual Question Answering: Elevate Your Multimodal Language Model Using Knowledge-Enriched Prompts (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Visual question answering (VQA) is a task that requires an understanding of both the image and the question to provide a natural language answer. |
| Approach: | They propose a multimodal framework that leverages language guidance to answer questions more accurately. |
| Outcome: | The proposed framework improves on the multi-choice question-answering task using CLIP and BLIP models. |
Supervising Unsupervised Open Information Extraction Models (D19-1)
Copied to clipboard
| Challenge: | Existing supervised methods that use labeled training data are expensive and difficult to adapt to new domains. |
| Approach: | They propose a supervised open information extraction framework that leverages unsupervised Open IE systems and labeled data to improve system performance. |
| Outcome: | The proposed method outperforms existing supervised and unsupervised models by a significant margin. |
LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models (2023.emnlp-main)
Copied to clipboard
Zhiqiang Hu, Lei Wang, Yihuai Lan, Wanyu Xu, Ee-Peng Lim, Lidong Bing, Xing Xu, Soujanya Poria, Roy Lee
| Challenge: | Large language models (LLMs) have shown unprecedented performance across various tasks. |
| Approach: | They propose an easy-to-use framework that integrates adapters into LLMs . they evaluate adapters on 14 datasets from two different reasoning tasks . |
| Outcome: | The proposed framework can be used to fine-tune open-access language models with task-specific data and instruction data. |
Bridging Modalities: Enhancing Cross-Modality Hate Speech Detection with Few-Shot In-Context Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has developed models targeting specific modalities but lacks transferability between formats. |
| Approach: | They conduct extensive experiments using few-shot in-context learning with large language models to explore the transferability of hate speech detection between modalities. |
| Outcome: | The proposed model outperforms vision-language demonstrations in few-shot learning settings. |
Efficient Methods for Natural Language Processing: A Survey (2023.tacl-1)
Copied to clipboard
Marcos Treviso, Ji-Ung Lee, Tianchu Ji, Betty van Aken, Qingqing Cao, Manuel R. Ciosici, Michael Hassid, Kenneth Heafield, Sara Hooker, Colin Raffel, Pedro H. Martins, André F. T. Martins, Jessica Zosa Forde, Peter Milder, Edwin Simpson, Noam Slonim, Jesse Dodge, Emma Strubell, Niranjan Balasubramanian, Leon Derczynski, Iryna Gurevych, Roy Schwartz
| Challenge: | Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data, but using only scale to improve performance means resource consumption also grows. |
| Approach: | They propose to use data, time, storage, or energy to improve model performance. |
| Outcome: | The proposed methods and findings provide guidance for conducting NLP under limited resources and point towards promising research directions for developing more efficient methods. |
Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing image instruction fine-tuning datasets do not fully exploit visual information to enhance multimodal reasoning capabilities of Large language models (LLMs). |
| Approach: | They propose a LLaVA-based model fine-tuned with MathV360K to bridge this gap by collecting 40K high-quality images with question-answer pairs from 24 existing datasets and synthesizing 320K new pairs. |
| Outcome: | The proposed model improves the multimodal reasoning capabilities of LLaVA-1.5 and demonstrates enhanced generalizability on the MMMU benchmark. |
Adapter-TST: A Parameter Efficient Method for Multiple-Attribute Text Style Transfer (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies explore performing text style transfer on attributes like age, gender, formality, politeness, and formality. |
| Approach: | They propose a framework that freezes the pre-trained model’s original parameters and enables the development of a multiple-attribute text style transfer model. |
| Outcome: | The proposed model outperforms state-of-the-art models on sentiment transfer and multiple-attribute transfer tasks with significantly less computational resources. |
ToxiCloakCN: Evaluating Robustness of Offensive Language Detection in Chinese with Cloaking Perturbations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing large language models struggle with systematically perturbed data designed to evade detection mechanisms. |
| Approach: | They propose a large language model with homophonic substitutions and emoji transformations to test their models' robustness against cloaking perturbations. |
| Outcome: | The proposed model underperforms in detecting offensive content when perturbations are applied to Chinese language datasets. |
Who Wrote it and Why? Prompting Large-Language Models for Authorship Verification (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing AV techniques, including stylometric and deep learning, face limitations in terms of data requirements and lack of explainability. |
| Approach: | They propose a technique that leverages Large-Language Models (LLMs) to provide step-by-step stylometric explanation prompts to verify authorship. |
| Outcome: | The proposed technique outperforms state-of-the-art baselines, operates effectively with limited training data, and enhances interpretability through intuitive explanations. |
Recent Advances in Online Hate Speech Moderation: Multimodality and the Role of Large Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | HS is any communication demeaning a person or a group based on social or ethnic characteristics that undermines social harmony and individual safety . the recent Israel-Hamas conflict has escalated both anti-Muslim and anti-Semitic sentiments worldwide . |
| Approach: | They examine the role of large language models and large multimodal models in HS moderation . they examine how text, images, and audio interact to spread hate speech . |
| Outcome: | The findings highlight the need for solutions in low-resource settings and highlight the gaps in existing methods. |