Papers by Yuexin Wu
Token Dropping for Efficient BERT Pretraining (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to accelerate pretraining of transformer-based models are computationally expensive and degrade performance on downstream tasks. |
| Approach: | They propose a "token dropping" method to accelerate the pretraining of transformer-based models by 25% . they leverage the already built-in masked language modeling loss to identify unimportant tokens with practically no computational overhead. |
| Outcome: | The proposed method reduces the pretraining cost of BERT models by 25% while achieving similar overall performance on downstream tasks. |
MIND: A Multi-agent Framework for Zero-shot Harmful Meme Detection (2025.acl-long)
Copied to clipboard
| Challenge: | a rapid expansion of memes on social media highlights the need for effective methods to detect harmful content. |
| Approach: | They propose a multi-agent framework for zero-shot harmful meme detection that does not rely on annotated data. |
| Outcome: | The proposed framework outperforms existing zero-shot approaches on three meme datasets. |
Unsupervised Reinforcement Adaptation for Class-Imbalanced Text Classification (2022.starsem-1)
Copied to clipboard
| Challenge: | Existing models for class imbalanced labels learn domain-invariant representations across domains and evaluate primarily on class-balanced data. |
| Approach: | They propose an unsupervised domain adaptation approach that leverages feature variants and imbalanced labels across domains to learn robust representations. |
| Outcome: | The proposed method can learn robust domain-invariant representations and adapt classifiers on imbalanced classes over domains. |
Multi-step Problem Solving Through a Verifier: An Empirical Analysis on Model-induced Process Supervision (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a method for process supervision has shown significant improvements in multi-step problem solving . despite the advances in process supervision, there are still easily observable mistakes in state-of-the-art LLMs. |
| Approach: | They propose a method for automating data curation by using a trained verifier to evaluate intermediate steps generated by a reasoner. |
| Outcome: | The proposed method improves the performance of PaLM 2 on math and coding tasks. |
Distilling Text Style Transfer With Self-Explanation From LLMs (2024.naacl-srw)
Copied to clipboard
| Challenge: | Text Style Transfer (TST) aims to alter the style of text while preserving its core content. |
| Approach: | They propose a framework that leverages large language models alongside chain-of-thought prompting to facilitate TST. |
| Outcome: | The proposed framework surpasses supervised fine-tuning and knowledge distillation methods in low-resource settings. |
Large Language Models Can Self-Improve (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have excellent performance in various tasks, but fine-tuning requires extensive supervision. |
| Approach: | They propose to use a pre-trained Large Language Model to generate rationale-augmented answers for unlabeled questions and fine-tune the LLM using those self-generated solutions as target outputs. |
| Outcome: | The proposed approach improves the general reasoning ability of a 540B-parameter LLM without any ground truth label. |
Unsupervised Cross-lingual Transfer of Word Embedding Spaces (D18-1)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual word mapping require cross-linguistic supervision, but this is not available for many low resource languages. |
| Approach: | They propose an unsupervised method that learns transformation functions over corresponding word embedding spaces using a distributed distributional matching algorithm. |
| Outcome: | The proposed method performs better on bilingual lexicon induction and cross-lingual word similarity prediction datasets than other supervised and unsupervised methods. |