Papers by Mengzhao Jia
AutoRubric: Rubric-Based Generative Rewards for Faithful Multimodal Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Multimodal large language models (MLLMs) have advanced from perception tasks to complex multi-step reasoning. |
| Approach: | They propose a framework that integrates reinforcement learning with verifiable rewards with process-level supervision through automatically collected rubric-based generative rewards. |
| Outcome: | The proposed framework achieves state-of-the-art performance on six multimodal reasoning benchmarks and significantly improves reasoning faithfulness in dedicated evaluations. |
Multi-source Semantic Graph-based Multimodal Sarcasm Explanation Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies can only identify sarcastic post but could not give concrete explanation for why it is sarkastic. |
| Approach: | They propose a multimodal sarcasm explanation scheme that generates a sentence for a social post to explain why it contains sarkasis. |
| Outcome: | The proposed model outperforms existing methods on a public dataset. |
Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) and Multimodal Large Language models (MLLMs) trained on vast web corpora can memorize and disclose individuals’ confidential and private data, raising legal and ethical concerns. |
| Approach: | They propose a benchmark to assess unlearning algorithms from multiple perspectives and provide a baseline for existing generative models. |
| Outcome: | The proposed benchmark consists of 500 fictitious profiles and 153 profiles for public celebrities, evaluated from both multimodal (image+text) and unimodal (text) perspectives. |
MultiChartQA: Benchmarking Vision-Language Models on Multi-Chart Problems (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing chart understanding benchmarks focus on single-chart tasks, neglecting multi-hop reasoning required to extract and integrate information from multiple charts. |
| Approach: | They propose a benchmark that evaluates MLLMs’ capabilities in four key areas: direct question answering, parallel question answering and comparative reasoning. |
| Outcome: | The proposed benchmark evaluates MLLMs’ capabilities in four key areas: direct question answering, parallel question answering and comparative reasoning. |
MMTutorBench: The First Multimodal Benchmark for AI Math Tutoring (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for AI math tutoring largely overlook these skills. |
| Approach: | They evaluate 12 leading multimodal large language models and find clear performance gaps between them. |
| Outcome: | The proposed benchmarks show that they can solve 770 problems and provide diagnostics and guidance to students step by step. |
PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuning (2024.acl-long)
Copied to clipboard
| Challenge: | Instruction tuning has advanced large language models (LLMs) but its application in lower-resource languages faces challenges due to the imbalanced foundational abilities of LLMs across different languages. |
| Approach: | They propose a pivot language guided generation approach that utilizes a high-resource language as the pivot to enhance instruction tuning in lower-resourced languages. |
| Outcome: | The proposed approach improves instruction-following abilities of LLMs by 29% on average compared to directly responding in the target language alone. |
Learn Beyond The Answer: Training Language Models with Reflection for Mathematical Reasoning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies focus on *broadening* the training set with data augmentation techniques to maximize such benefits. |
| Approach: | They propose a method that embeds problem reflection into each training instance. |
| Outcome: | The proposed method enhances performance in standard and complex scenarios that require reflective thinking. |