Papers by Belinda Zeng
DynaMaR: Dynamic Prompt with Mask Token Representation (2022.emnlp-industry)
Copied to clipboard
Xiaodi Sun, Sunny Rajagopalan, Priyanka Nigam, Weiyi Lu, Yi Xu, Iman Keivanloo, Belinda Zeng, Trishul Chilimbi
| Challenge: | Recent research shows that large language models pretrained using unsupervised approaches can achieve significant performance improvement on many downstream tasks. |
| Approach: | They propose an unsupervised approach to fine-tuning large language models using unsupervised approaches to many downstream tasks. |
| Outcome: | The proposed approach improves on four e-commerce applications and can achieve an average improvement of 10% in few-shot settings and 3.7% in data-rich settings over the standard approach. |
Learning LLM Preference over Intra-Dialogue Pairs: A Framework for Utterance-level Understandings (2025.naacl-industry)
Copied to clipboard
Xuanqing Liu, Luyang Kong, Wei Niu, Afshin Khashei, Belinda Zeng, Steve Johnson, Jon Jay, Davor Golac, Matt Pope
| Challenge: | Large language models (LLMs) have demonstrated remarkable capabilities in handling complex dialogue tasks without requiring use case-specific fine-tuning. |
| Approach: | They propose a framework that combines the scalability of LLM-generated labels with the precision of human annotations to achieve higher speed and accuracy comparable to larger models. |
| Outcome: | The proposed framework significantly improves accuracy across utterance-level dialogue tasks, including sentiment detection (over 2%), dialogue act classification (over 1.5%), etc. |
OssCSE: Overcoming Surface Structure Bias in Contrastive Learning for Unsupervised Sentence Embedding (2023.emnlp-main)
Copied to clipboard
Zhan Shi, Guoyin Wang, Ke Bai, Jiwei Li, Xiang Li, Qingjun Cui, Belinda Zeng, Trishul Chilimbi, Xiaodan Zhu
| Challenge: | Recent studies show that contrastive learning is effective in sentence representation learning . but, the surface structure bias is a problem in the current model . |
| Approach: | They propose to combine a sentence with a sub-semantic sentence to investigate the surface structure bias. |
| Outcome: | The proposed model achieves state-of-the-art on standard semantic textual similarity tasks using different pre-trained backbones. |
Asynchronous Convergence in Multi-Task Learning via Knowledge Distillation from Converged Tasks (2022.naacl-industry)
Copied to clipboard
Weiyi Lu, Sunny Rajagopalan, Priyanka Nigam, Jaspreet Singh, Xiaodi Sun, Yi Xu, Belinda Zeng, Trishul Chilimbi
| Challenge: | Multi-task learning (MTL) aims to solve multiple tasks by sharing a base representation among them. |
| Approach: | They propose an approach that allows for "asynchronous" convergence among the tasks where each task can converge on its own schedule. |
| Outcome: | The proposed method outperforms existing methods in two 5-task MTL setups. |
ReAugKD: Retrieval-Augmented Knowledge Distillation For Pre-trained Language Models (2023.acl-short)
Copied to clipboard
Jianyi Zhang, Aashiq Muhamed, Aditya Anantharaman, Guoyin Wang, Changyou Chen, Kai Zhong, Qingjun Cui, Yi Xu, Belinda Zeng, Trishul Chilimbi, Yiran Chen
| Challenge: | Knowledge distillation (KD) is an effective compression technique to derive a smaller student model from a larger teacher model by transferring the knowledge embedded in the teacher's network. |
| Approach: | They propose a framework and loss function that preserves the semantic similarities of teacher and student training examples to enable the student to retrieve from the knowledge base effectively. |
| Outcome: | The proposed framework preserves the semantic similarities of teacher and student training examples to achieve state-of-the-art performance on the GLUE benchmark. |