Papers by Hyunju Lee
TransferCVLM: Transferring Cross-Modal Knowledge for Vision-Language Modeling (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent large vision-language multimodal models pre-trained with huge amount of image-text pairs show remarkable performances in downstream tasks. |
| Approach: | They propose a method of efficient knowledge transfer that integrates pre-trained uni-modal models into a combined vision-language model without pre-training . they propose to fine-tune the model and transfer multimodal knowledge from a teacher vision-linguistic model to the CVLM for each task application. |
| Outcome: | The proposed method outperforms existing vision-language models in downstream tasks. |
Extracting Chemical-Protein Interactions via Calibrated Deep Neural Network and Self-training (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Several natural language processing methods have been used to extract interactions between chemicals and proteins from biomedical text data. |
| Approach: | They propose a method to extract chemical–protein interactions from biomedical text data . they use a pre-trained language-understanding model and calibration techniques to estimate uncertainty . |
| Outcome: | The proposed approach achieves state-of-the-art performance on the Biocreative VI ChemProt task while preserving higher calibration abilities. |
GENDEX: Generative Data Augmentation Strategy Leveraging External Data for Abstractive Dialogue Summarization (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to summarize text data are limited by the lack of data. |
| Approach: | They propose a method that uses external data to generate synthetic dialogues from short texts containing people and their interpersonal interactions. |
| Outcome: | The proposed method shows robust performance, generalizability, and scalability regardless of complexity of dialogues. |
Early Stopping Based on Unlabeled Samples in Text Classification (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to stop models from overfitting are based on a separate validation set, but in low resource settings, a small validation set may not be representative enough. |
| Approach: | They propose a method that uses unlabeled samples to estimate the class distribution of the unlabed samples. |
| Outcome: | The proposed method performs better than existing stop-methods in balanced and imbalanced data settings. |
Domain Knowledge Transferring for Pre-trained Language Model via Calibrated Activation Boundary Distillation (2022.acl-long)
Copied to clipboard
| Challenge: | Pretrained language models are used to boost their performance on downstream tasks . pretraining with in-domain texts requires considerable in- domain data and training resources . |
| Approach: | They propose a domain knowledge transferring framework for pre-trained language models without additional in-domain pretraining. |
| Outcome: | The proposed framework extracts domain knowledge from an existing in-domain pretrained language model and transfers it to other PLMs by applying knowledge distillation. |