AFMRL: Attribute-Enhanced Fine-Grained Multi-Modal Representation Learning in E-commerce (2026.findings-acl)
Copied to clipboard
| Challenge: | Multimodal representation is crucial for E-commerce tasks such as identical product retrieval. |
| Approach: | They propose an approach which leverages the generative power of Multimodal Large Language Models to extract key attributes from product images and text and enhances representation learning through a two-stage training framework. |
| Outcome: | The proposed model achieves state-of-the-art on multiple downstream retrieval tasks, validating the effectiveness of harnessing generative models to advance fine-grained representation learning. |
Similar Papers
Large Scale Generative Multimodal Attribute Extraction for E-commerce Attributes (2023.acl-industry)
Copied to clipboard
| Challenge: | E-commerce websites often don’t label or mislabel attributes of products . |
| Approach: | They propose a multi-modal product attribute generation system that extracts product attributes from the product pages of eCommerce stores by using both text and images. |
| Outcome: | The proposed model improves the recall@90P accuracy by 10.16% and 6.9 from the state-of-the-art models. |
Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding (2026.acl-long)
Copied to clipboard
Sensen Gao, Shanshan Zhao, Xu Jiang, Lunhao Duan, Yong Xien Chng, Qing-Guo Chen, Weihua Luo, Kaifu Zhang, Jia-Wang Bian, Mingming Gong
| Challenge: | Document understanding is critical for applications from financial analysis to scientific discovery. |
| Approach: | They propose a taxonomy based on domain, retrieval modality, and granularity and review advances involving graph structures and agentic frameworks. |
| Outcome: | The proposed model enables holistic retrieval and reasoning across all modalities, unlocking comprehensive document intelligence. |
Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation (2025.findings-acl)
Copied to clipboard
Mohammad Mahdi Abootorabi, Amirhosein Zobeiri, Mahdi Dehghani, Mohammadali Mohammadkhani, Bardia Mohammadi, Omid Ghahroodi, Mahdieh Soleymani Baghshah, Ehsaneddin Asgari
| Challenge: | Large Language Models (LLMs) suffer from hallucinations and outdated knowledge due to their reliance on static training data. |
| Approach: | They review training strategies, robustness enhancements, loss functions, and agent-based approaches and outline open challenges and future directions to guide research in this evolving field. |
| Outcome: | The proposed model improves accuracy and accuracy while integrating external dynamic information for improved factual grounding. |
Generative Cross-Modal Retrieval: Memorizing Images in Multimodal Language Models for Retrieval and Beyond (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in generative language models have demonstrated their ability to memorize knowledge from documents and recall knowledge to respond to user queries effectively. |
| Approach: | They propose to enable multimodal large language models to memorize and recall images within their parameters. |
| Outcome: | The proposed model performs well even with large-scale image candidate sets. |
Retrieving Multimodal Information for Augmented Generation: A Survey (2023.findings-emnlp)
Copied to clipboard
Ruochen Zhao, Hailin Chen, Weishi Wang, Fangkai Jiao, Xuan Long Do, Chengwei Qin, Bosheng Ding, Xiaobao Guo, Minzhi Li, Xingxuan Li, Shafiq Joty
| Challenge: | Large Language Models (LLMs) are increasingly using multimodality to augment their generation ability, but there is no unified perception of at which stage and how to incorporate different modalities. |
| Approach: | They propose to use multimodality to augment Large Language Models (LLMs) this will provide scholars with a deeper understanding of the methods' applications and encourage them to adapt existing techniques to the fast-growing field of LLMs. |
| Outcome: | The proposed methods improve factuality, reasoning, interpretability, and robustness of the generated content. |
Open-World Attribute Mining for E-Commerce Products with Multimodal Self-Correction Instruction Tuning (2025.acl-long)
Copied to clipboard
| Challenge: | Current AM methods focus on extracting attributes from unimodal text, underutilizing multimodal data. |
| Approach: | They propose a framework for multimodal self-correction instruction tuning to extract new attributes from images and text with Multimodal Large Language Models. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two datasets. |
MMEvol: Empowering Multimodal Large Language Models with Evol-Instruct (2025.findings-acl)
Copied to clipboard
Run Luo, Haonan Zhang, Longze Chen, Ting-En Lin, Xiong Liu, Yuchuan Wu, Min Yang, Yongbin Li, Minzheng Wang, Pengpeng Zeng, Lianli Gao, Heng Tao Shen, Yunshui Li, Hamid Alinejad-Rokny, Xiaobo Xia, Jingkuan Song, Fei Huang
| Challenge: | a new framework for image-text instruction data evolution improves MLLM performance . lack of high-quality instruction data remains a major bottleneck in ML modeling . |
| Approach: | They propose a multimodal instruction data evolution framework that iteratively enhances data quality through fine-grained perception, cognitive reasoning, and interaction evolution. |
| Outcome: | The proposed approach improves MLLM performance in nine vision-language tasks while using significantly less data. |
MDocRAG-RL: Empowering Multi-Modal Document RAG via Complex Visual Reasoning with Reinforcement Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing RAG systems produce suboptimal embeddings and naively insert images into context without adequate visual perception, limiting reasoning capabilities. |
| Approach: | They propose a novel RAG framework for complex visual reasoning that integrates multimodal large language models with external knowledge to enhance retrieval efficiency. |
| Outcome: | The proposed framework achieves state-of-the-art performance on multiple benchmarks. |
CART: A Generative Cross-Modal Retrieval Framework With Coarse-To-Fine Semantic Modeling (2025.acl-long)
Copied to clipboard
Minghui Fang, Shengpeng Ji, Jialong Zuo, Hai Huang, Yan Xia, Jieming Zhu, Xize Cheng, Xiaoda Yang, Wenrui Liu, Gang Wang, Zhenhua Dong, Zhou Zhao
| Challenge: | Cross-modal retrieval tasks are used to retrieve data from one modality or another based on a query from another modality. |
| Approach: | They propose a generative cross-modal retrieval framework based on coarse-to-fine semantic modeling . they propose combining K-Means and RQ-VAE to discretize multimodal data into token sequences that support autoregressive generation. |
| Outcome: | The proposed framework achieves excellent performance and efficiency in multimodal retrieval tasks. |
MMAPS: End-to-End Multi-Grained Multi-Modal Attribute-Aware Product Summarization (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing product summarization methods lack end-to-end product summaries and multi-grained multi-modal modeling. |
| Approach: | They propose an end-to-end multi-grained multi-modal attribute-aware product summarization method that jointly models product attributes and generates product summaries. |
| Outcome: | The proposed method outperforms state-of-the-art product summarization methods on a large-scale Chinese e-commence dataset. |