Papers by Sicheng Song
PhysicsArena: The First Multimodal Physics Reasoning Benchmark Exploring Variable, Process, and Solution Dimensions (2025.findings-emnlp)
Copied to clipboard
Song Dai, Yibo Yan, Jiamin Su, Zihao Dongfang, Yubo Gao, Yonghua Hei, Jungang Li, Junyan Zhang, Sicheng Tao, Zhuoran Gao, Xuming Hu
| Challenge: | Current physics benchmarks focus on text-only inputs or only on problem-solving . current physics reasoning benchmarks neglect critical intermediate steps of variable identification and process formulation. |
| Approach: | a new benchmark evaluates multimodal large language models in physics reasoning . the benchmark measures variables, process formulations, and solution derivation . |
| Outcome: | PhysicsArena is the first multimodal physics reasoning benchmark . it evaluates MLLMs across three critical dimensions: variable identification, process formulation, and solution derivation. |
Both Text and Images Leaked! A Systematic Analysis of Data Contamination in Multimodal LLM (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for unimodal large language models are inadequate for MLLMs due to multimodal data complexity and multi-phase training. |
| Approach: | MM-DETECT analyzes data contamination using a framework that defines two contamination categories - unimodal and cross-modal . |
| Outcome: | The proposed framework quantifies contamination severity across multiple-choice and caption-based Visual Question Answering tasks. |
Unmasking Deceptive Visuals: Benchmarking Multimodal Large Language Models on Misleading Chart Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Misleading visualizations can distort perception and lead to incorrect conclusions. |
| Approach: | They propose a large-scale multimodal dataset to evaluate MLLMs on misleading chart reasoning. |
| Outcome: | The proposed framework evaluates MLLMs on misleading chart reasoning on a large-scale multimodal dataset spanning 21 misleader types and 10 chart types . it contains 3,026 curated examples spanning standard chart code, CSV data, multiple-choice questions, and labeled explanations, validated through iterative MLML checks and exhausted expert human review. |
FPC: Fine-tuning with Prompt Curriculum for Relation Extraction (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing methods for relation extraction ignore semantics of relation labels . prompt-based fine-tuning has been proposed for RE . |
| Approach: | They propose a method for relation extraction using prompt-based fine-tuning . they use auxiliary prompt-tuned learning task to make the model capture semantics of relation labels . |
| Outcome: | The proposed method outperforms existing methods on four widely used RE benchmarks under fully supervised and low-resource settings. |