Papers by Rong Jin
SumCSE: Summary as a transformation for Contrastive Learning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Sentence embedding models are typically trained using contrastive learning (CL) using human annotations directly or by repurposing other annotated datasets. |
| Approach: | They propose to use generative language models to generate CL data using annotated data. |
| Outcome: | The proposed method outperforms the previous best unsupervised method by 1.8 points and SimCSE, a strong supervised baseline by 0.3 points on the semantic text similarity (STS) benchmark. |
ReasonRec: A Reasoning-Augmented Multimodal Agent for Unified Recommendation (2026.findings-acl)
Copied to clipboard
Yihua Zhang, Mingfu Liang, Jiyan Yang, Rong Jin, Wen-Yen Chen, Yiping Han, Huayu Li, Buyun Zhang, Liang Luo, Luke Simon, Sijia Liu, Tianlong Chen, Xi Liu
| Challenge: | Recent advances in multimodal recommenders lack explicit reasoning and self-awareness of uncertainty. |
| Approach: | They propose a reasoning-augmented multimodal agent structured around a three-stage explicit reasoning pipeline. |
| Outcome: | The proposed agent improves ranking metrics and performance on four standard recommendation tasks across five real-world datasets. |
Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) tasks require large amounts of parallel data to augment training. |
| Approach: | They propose a data augmentation paradigm that augments each training instance with an adjacency semantic region that could cover adequate variants of literal expression under the same meaning. |
| Outcome: | The proposed paradigm improves on the state-of-the-art in supervised neural machine translation tasks. |
UNICORN: A Unified Causal Video-Oriented Language-Modeling Framework for Temporal Video-Language Tasks (2024.emnlp-main)
Copied to clipboard
Yuanhao Xiong, Yixin Nie, Haotian Liu, Boxin Wang, Jun Chen, Rong Jin, Cho-Jui Hsieh, Lorenzo Torresani, Jie Lei
| Challenge: | Recent advances in large multimodal models have encouraged the development of large multi-modal models . however, it is unclear how to extend these models to the more complex video domain . |
| Approach: | They propose a visual instruction tuning framework to address temporal video-language tasks . they collect a dataset and fine-tune the framework on instruction-following data . |
| Outcome: | The proposed model can perform better on established temporal video-language tasks without training objectives and intensive pre-training. |
Scaling Law for Multimodal Large Language Model Supervised Fine-Tuning (2026.acl-long)
Copied to clipboard
YiFan Zhang, Tao Yu, Feng Li, Chaoyou Fu, Yibo Hu, Kun Wang, Qingsong Wen, Zhang Zhang, Liang Wang, Rong Jin
| Challenge: | supervised fine-tuning (SFT) is crucial for multimodal large language models, yet a comprehensive scaling law is lacking . et al.: scaling laws focus on model size, pre-training tokens, and MLLM SFT data volumes . |
| Approach: | They propose two scaling laws to guide optimal model-data configuration . they propose one applicable when training data volumes are well defined by researchers . |
| Outcome: | The proposed scaling laws provide valuable recommendations for optimal resource allocation . they show that the proposed laws are more accurate than existing models . |