Papers by Chi Sun
Model Composition for Multimodal Large Language Models (2024.acl-long)
Copied to clipboard
Chi Chen, Yiyang Du, Zheng Fang, Ziyue Wang, Fuwen Luo, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Maosong Sun, Yang Liu
| Challenge: | Existing methods for creating versatile MLLMs rely on joint training with paired instruction data, which is resource-intensive and challenging to extend to new modalities. |
| Approach: | They propose a new paradigm for multimodal large language models by reusing modality encoders and merging LLM parameters. |
| Outcome: | The proposed model retains the modal understanding capabilities of each original model. |
RSMeM: Knowledge-Enhanced Memory Evolution for Remote Sensing Agents with Systematic Evaluation (2026.acl-long)
Copied to clipboard
Bingxian Wu, Yu Zhang, Zonghao Guo, Tang Liu, Chen Qian, Yuxiang Lu, Xingbo Du, Yanghao Li, Yidan Zhang, Chi Chen, Ling Yao, Chenghu Zhou, Maosong Sun
| Challenge: | Existing RS agents built on general-purpose LLMs are domain-agnostic, resulting in brittle and error-prone workflows. |
| Approach: | They propose a knowledge-enhanced memory evolution mechanism that bootstraps RS agents with pre-distilled domain knowledge and iteratively integrates online experience for robust multi-step tool execution. |
| Outcome: | Experiments show that the new model improves tool-use performance and accuracy . iteratively, iteration of the model integrates online experience for robust multi-step tool execution . |
Question Generation from SQL Queries Improves Neural Semantic Parsing (D18-1)
Copied to clipboard
| Challenge: | Using question generation, we learn a semantic parser with 30% of the supervised training data. |
| Approach: | They propose to use question generation to learn a semantic parser with less supervised training data. |
| Outcome: | The proposed method improves the state-of-the-art model with less training data. |
LaMP-Val: Large Language Models Empower Personalized Valuation in Auction (2025.findings-emnlp)
Copied to clipboard
Jie Sun, Tianyu Zhang, Houcheng Jiang, Kexin Huang, Xiang Shu, Zhibo Zhu, Lintao Ma, Xingyu Lu, Jun Zhou, Junkang Wu, Chi Luo, An Zhang, Jiancan Wu, Xiang Wang
| Challenge: | Currently, most research focuses on the bidding algorithms used within auction mechanisms. |
| Approach: | They propose a personalized valuation framework that integrates Large Language Models to incorporate personalized semantic preference into users valuation process. |
| Outcome: | The proposed framework incorporates Large Language Models to incorporate personalized semantic preference into users valuation process. |
Browse and Concentrate: Comprehending Multimodal Content via Prior-LLM Context Fusion (2024.acl-long)
Copied to clipboard
Ziyue Wang, Chi Chen, Yiqi Zhu, Fuwen Luo, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Maosong Sun, Yang Liu
| Challenge: | Multimodal Large Language Models (MLLMs) lack understanding of multi-image and interleaved inputs due to the visual features encoded by frozen encoders before being fed into the LLM backbone. |
| Approach: | They propose a two phase paradigm to enable in-depth multimodal context fusion prior to feeding the features into LLMs. |
| Outcome: | The proposed paradigm boosts the performance on 7 multi-image scenarios, contributing to increments on average accuracy by 2.13% and 7.60% against strong MLLMs baselines with 3B and 11B LLMs, respectively. |
ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing open-source MLLMs fail to fully capture dense information embedded in charts . current models still face significant challenges in understanding and analyzing visual tasks such as captioning and question answering. |
| Approach: | They propose a chart-to-code MLLM which leverages Code LLMs as the language backbone to enhance the executability of the generated code. |
| Outcome: | The proposed model surpasses existing open-source models on chart-to-code benchmarks with only 7B parameters and provides lossless representations that contain all critical details. |
AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning (2025.emnlp-demos)
Copied to clipboard
Zhong Zhang, Yaxi Lu, Yikun Fu, Yupeng Huo, Shenzhi Yang, Yesai Wu, Han Si, Xin Cong, Haotian Chen, Yankai Lin, Xie Xie, Wei Zhou, Wang Xu, Zhou Su, Zhongwu Zhai, Xiaoming Liu, null Meiyudong, Jianming Xu, Hongyan Tian, Chongyi Wang, Chi Chen, Yuan Yao, Zhiyuan Liu, Maosong Sun
| Challenge: | Large language model agents have enabled GUI-based automation, but their deployment is limited by noisy data, poor generalization, and lack of support for non-English GUIs. |
| Approach: | They propose an 8B-parameter GUI agent built for robust and efficient on-device GUI interaction. |
| Outcome: | The proposed GUI agent achieves promising performance on five public benchmarks and proposed Chinese benchmark CAGUI. |
Efficient Pretraining Data Selection for Language Models via Multi-Actor Collaboration (2025.acl-long)
Copied to clipboard
Tianyi Bai, Ling Yang, Zhen Hao Wong, Fupeng Sun, Xinlin Zhuang, Jiahui Peng, Chi Zhang, Lijun Wu, Qiu Jiantao, Wentao Zhang, Binhang Yuan, Conghui He
| Challenge: | Efficient data selection is crucial to accelerate the pretraining of language models . limited research has addressed the inherent conflicts between data selection methods . |
| Approach: | They propose a multi-actor collaborative data selection mechanism that prioritizes data based on its specific criterion and updates prioritization rules using the current state of the model. |
| Outcome: | The proposed model accelerates convergence in LM pretraining and achieves an average relative performance gain of 10.5% across multiple language model benchmarks. |
End-to-End Unsupervised Vision-and-Language Pre-training with Referring Expression Matching (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing unsupervised vision-and-language pre-training methods take pre-extracted region-based visual features from external object detectors, which limits flexibility and reduces computational efficiency. |
| Approach: | They propose an unsupervised vision-and-language pre-training task that predicts which patches contain an object referred to in natural language from the encoded visual features. |
| Outcome: | The proposed approach outperforms existing methods and obtains state-of-the-art results on four vision-and-language tasks. |
Can LMs Generalize to Future Data? An Empirical Analysis on Text Summarization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing summarization benchmarks overlap in time with pre-training corpora and fine-tuning datasets. |
| Approach: | They propose a temporal generalization benchmark that contains data samples from 2010 to 2022 to understand the temporal ability of abstractive summarization models. |
| Outcome: | The proposed benchmark analyzes data samples from 2010 to 2022 to understand the temporal generalization ability of abstractive summarization models. |
Mask-Align: Self-Supervised Neural Word Alignment (2021.acl-long)
Copied to clipboard
| Challenge: | Word alignment is an important task in many natural language processing tasks. |
| Approach: | They propose a self-supervised word alignment model that takes advantage of the full context on the target side. |
| Outcome: | The proposed model outperforms previous unsupervised models and obtains state-of-the-art results on four language pairs. |
CODIS: Benchmarking Context-dependent Visual Comprehension for Multimodal Large Language Models (2024.acl-long)
Copied to clipboard
Fuwen Luo, Chi Chen, Zihao Wan, Zhaolu Kang, Qidong Yan, Yingjie Li, Xiaolong Wang, Siyu Wang, Ziyue Wang, Xiaoyue Mi, Peng Li, Ning Ma, Maosong Sun, Yang Liu
| Challenge: | Multimodal large language models have demonstrated promising results in a variety of tasks that combine vision and language. |
| Approach: | They propose a benchmark to assess the ability of models to use contextual information in free-form text to enhance visual comprehension. |
| Outcome: | The proposed model fails to extract and utilize contextual information to improve understanding of images. |
User Feedback Alignment for LLM-powered Exploration in Large-scale Recommendation Systems (2025.acl-industry)
Copied to clipboard
Jianling Wang, Yifan Liu, Yinghao Sun, Xuejian Ma, Yueqi Wang, He Ma, Zhengyang Su, Minmin Chen, Mingyan Gao, Onkar Dalal, Ed H. Chi, Lichan Hong, Ningren Han, Haokai Lu
| Challenge: | Large Language Models (LLMs) can be used to broaden user experiences beyond established preferences and reinforce feedback loops. |
| Approach: | They propose a hierarchical approach that combines hierarchic planning with LLM inference-time scaling to improve recommendation relevancy without compromising novelty. |
| Outcome: | The proposed approach shows significant gains in both user satisfaction and exploration diversity. |
EchoMLLM: Incentivizing Echocardiographic Video Understanding with Keyframe Grounding and Report Generation (2026.findings-acl)
Copied to clipboard
Heyu Huang, Wanran Sun, Chi Chen, Bo Chen, Zonghao Guo, Yuhua Li, Ruixuan Li, Kunlun He, Maosong Sun
| Challenge: | Echocardiography analysis requires a dual capability: rigorous quantitative keyframe localization and comprehensive qualitative synthesis. |
| Approach: | They propose a unified framework designed for real-world echocardiography video understanding. |
| Outcome: | a new framework is designed to support real-world echocardiography video understanding . it reduces temporal grounding errors by up to 76% and improves report generation quality by 65% . |
VCWE: Visual Character-Enhanced Word Embeddings (N19-1)
Copied to clipboard
| Challenge: | Currently, word embeddings are playing a pivotal role in many natural language processing tasks. |
| Approach: | They propose a model to learn Chinese word embeddings via three-level composition . they use convolutional neural network to extract intra-character compositionality from character shape . |
| Outcome: | The proposed model performs better on word similarity, sentiment analysis, named entity recognition and part-of-speech tagging tasks. |
GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge (D19-1)
Copied to clipboard
| Challenge: | Word Sense Disambiguation (WSD) aims to find the exact sense of an ambiguous word in a particular context. |
| Approach: | They propose to integrate gloss knowledge into supervised neural networks for Word Sense Disambiguation (WSD) this paper proposes to fine-tune a pre-trained BERT model and achieve new state-of-the-art results on WSD task. |
| Outcome: | The proposed model achieves state-of-the-art on the word Sense Disambiguation (WSD) task. |
Migician: Revealing the Magic of Free-Form Multi-Image Grounding in Multimodal Large Language Models (2025.findings-acl)
Copied to clipboard
You Li, Heyu Huang, Chi Chen, Kaiyu Huang, Chao Huang, Zonghao Guo, Zhiyuan Liu, Jinan Xu, Yuhua Li, Ruixuan Li, Maosong Sun
| Challenge: | Existing MLLMs still struggle to achieve precise grounding in multi-image scenarios. |
| Approach: | They propose a Chain-of-Thought framework that integrates single-image grounding with multi-image comprehension to address this challenge. |
| Outcome: | The proposed model outperforms existing models in multi-image grounding tasks by 24.94% and surpasses larger 70B models. |
Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence (N19-1)
Copied to clipboard
| Challenge: | Sentiment analysis (SA) is a computational task that aims to identify opinion polarity towards a specific aspect. |
| Approach: | They propose to convert ABSA into a sentence-pair classification task such as question answering and natural language inference. |
| Outcome: | The proposed model is fine-tuned and achieves state-of-the-art on SentiHood and SemEval-2014 datasets. |
Weakly Supervised Vision-and-Language Pre-training with Relative Representations (2023.acl-long)
Copied to clipboard
| Challenge: | Weakly supervised vision-and-language pre-training (WVLP) uses only local descriptions of images as cross-modal anchors to construct weakly-aligned image-text pairs for pre- training. |
| Approach: | They propose to take a small number of aligned image-text pairs as anchors and represent each unaligned image and text by its similarities to these anchors. |
| Outcome: | The proposed model reduces the cost of pre-training while maintaining decent performance on downstream tasks. |
FinHEAR: Human Expertise and Adaptive Risk-Aware Temporal Reasoning for Financial Decision-Making (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models often overlook key behavioral patterns underlying human financial behavior. |
| Approach: | FinHEAR is a multi-agent framework for human expertise and Adaptive Risk-aware reasoning. |
| Outcome: | FinHEAR outperforms baseline models in trend forecasting and decision-making. |