MathFusion: Enhancing Mathematical Problem-solving of LLM through Instruction Fusion (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive progress in mathematical problem-solving . current approaches to enhance mathematical reasoning focus on instance-level modifications . |
| Approach: | They propose a framework that enhances mathematical reasoning through cross-problem instruction synthesis. |
| Outcome: | The proposed framework boosts mathematical reasoning by 18.0 points while maintaining high data efficiency. |
Similar Papers
FLAMES: Improving LLM Math Reasoning via a Fine-Grained Analysis of the Data Synthesis Pipeline (2025.findings-emnlp)
Copied to clipboard
Parker Seegmiller, Kartik Mehta, Soumya Saha, Chenyang Tao, Shereen Oraby, Arpit Gupta, Tagyoung Chung, Mohit Bansal, Nanyun Peng
| Challenge: | Recent work improving LLM math reasoning with synthetic data uses unique setups, making comparison of data synthesis strategies impractical. |
| Approach: | They propose a framework for LLM assessment of math reasoning with synthetic data . they use 10 existing data synthesis strategies and multiple other factors to study performance . |
| Outcome: | The proposed data synthesis strategies outperform public datasets on OlympiadBench, CollegeMath, GSMPlus and MATH. |
Let’s Reason Formally: Natural-Formal Hybrid Reasoning Enhances LLM’s Math Capability (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has focused on improving the mathematical reasoning capabilities of Large Language Models (LLMs). |
| Approach: | They propose an end-to-end framework to integrate FL into NL math reasoning . they propose a problem alignment method that reformulates QA and existence problems . |
| Outcome: | The proposed framework achieves 89.80% and 84.34% accuracy rates on the MATH-500 and the AMC benchmarks. |
More Data or Better Data? A Critical Analysis of Data Selection and Synthesis for Mathematical Reasoning (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Despite various proposed data construction methods, their practical utility in real-world pipelines remains underexplored. |
| Approach: | They conduct a comprehensive analysis of open-source datasets and data synthesis techniques for mathematical reasoning under a unified pipeline designed to mirror training and deployment scenarios. |
| Outcome: | The proposed pipelines mirror training and deployment scenarios and are suitable for industrial applications. |
MathMixup: Boosting LLM Mathematical Reasoning with Difficulty-Controllable Data Synthesis and Curriculum Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing data synthesis methods suffer from limited diversity and lack precise control over problem difficulty, making them insufficient for efficient training paradigms such as curriculum learning. |
| Approach: | They propose a data synthesis paradigm that generates high-quality, difficulty-controllable mathematical reasoning problems through hybrid and decomposed strategies. |
| Outcome: | The proposed paradigm outperforms existing methods and improves mathematical reasoning abilities. |
ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline (2024.findings-emnlp)
Copied to clipboard
Yifan Xu, Xiao Liu, Xinghan Liu, Zhenyu Hou, Yueyan Li, Xiaohan Zhang, Zihan Wang, Aohan Zeng, Zhengxiao Du, Zhao Wenyi, Jie Tang, Yuxiao Dong
| Challenge: | Large language models (LLMs) have shown excellent mastering of human language but struggle in real-world applications that require mathematical problem-solving. |
| Approach: | They propose a pipeline to train a general Math-Critique model from the LLM itself to provide feedback signals and employ rejective fine-tuning and direct preference optimization over the Llm's own generations for data collection. |
| Outcome: | The proposed pipeline outperforms existing LLMs that could be two times larger. |
Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing image instruction fine-tuning datasets do not fully exploit visual information to enhance multimodal reasoning capabilities of Large language models (LLMs). |
| Approach: | They propose a LLaVA-based model fine-tuned with MathV360K to bridge this gap by collecting 40K high-quality images with question-answer pairs from 24 existing datasets and synthesizing 320K new pairs. |
| Outcome: | The proposed model improves the multimodal reasoning capabilities of LLaVA-1.5 and demonstrates enhanced generalizability on the MMMU benchmark. |
RV-Syn: Rational and Verifiable Mathematical Reasoning Data Synthesis based on Structured Function Library (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for generating high-quality reasoning data are limited in quality and availability. |
| Approach: | They propose a method that constructs mathematical operations and generates verifiable graphs that are back-translated into complex problems. |
| Outcome: | The proposed method achieves a 6.3% performance gain over existing methods on LLaMA-3-8B and outperforms others with only half the training data (50k vs. 100k). |
ClozeMath: Improving Mathematical Reasoning in Language Models by Learning to Fill Equations (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to train large language models do not capture how humans learn to think. |
| Approach: | They propose a method to fine-tune large language models for mathematical reasoning by using a text-infilling task that predicts masked equations from a given solution. |
| Outcome: | Experiments on GSM8K, MATH, and GSM-Symbolic show that ClozeMath surpasses baseline Masked Thought in performance and robustness with two test-time scaling decoding algorithms, Beam Search and Chain-of-Thought decoding. |
MARIO: MAth Reasoning with code Interpreter Output - A Reproducible Pipeline (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models lack mathematical reasoning, a hurdle on the path to true artificial general intelligence. |
| Approach: | They propose a protocol for fine-tuning large language models with a Python code interpreter to enhance the text analysis of the LLMs. |
| Outcome: | The proposed protocol improves the performance of a 7B-parameter LLM on the GSM8K and MATH datasets while allowing for an outlier-free value model-based inference method. |
MATHSENSEI: A Tool-Augmented Large Language Model for Mathematical Reasoning (2024.naacl-long)
Copied to clipboard
| Challenge: | TALMs have been successfully employed in question-answering benchmarks, but their efficacy on complex mathematical reasoning benchmarks are open research questions. |
| Approach: | They propose a tool-augmented large language model for mathematical reasoning that enhances the skillset of large language models (LLMs) by 13.5%. |
| Outcome: | The proposed model achieves better accuracy and better knowledge retrieval performance than existing tools. |