CADReview: Automatically Reviewing CAD Programs with Error Detection and Correction (2025.acl-long)
Copied to clipboard
| Challenge: | Computer-aided design (CAD) is crucial in prototyping 3D objects through geometric instructions. |
| Approach: | They propose a CAD review task to automatically detect and correct potential errors . they propose CAD program repairer framework to provide helpful feedback on error correction . |
| Outcome: | The proposed framework outperforms existing MLLMs in detecting errors and providing feedback on error correction. |
Similar Papers
CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | State-of-the-art large language models (LLMs) have demonstrated impressive code generation capabilities but struggle with real-world software engineering tasks such as revising source code to address code reviews. |
| Approach: | They propose a benchmark to evaluate large language models' ability to bridge both technical and conversational contexts by decomposing the generation task of code refinement into three essential reasoning steps. |
| Outcome: | The proposed benchmark exposes specific model weaknesses in code review comprehension disentangled from their generative automated code refinement results. |
CADMate: Generating CAD Assembly Plan with Geometric Chain-of-Thought and Spatial Physical Rewards (2026.acl-long)
Copied to clipboard
| Challenge: | Computer-aided design (CAD) is crucial in prototyping complex 3D objects . designers manually define assembly sequences for individual CAD parts . |
| Approach: | They propose a framework for computer-aided design that predicts actions for CAD parts . they use a reference design image and disassembled parts to generate 6-DoF transformations . |
| Outcome: | The proposed framework outperforms existing MLLMs in the design of CAD assemblies. |
VDebugger: Harnessing Execution Feedback for Debugging Visual Programs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Visual programs are executable code generated by large language models to address visual reasoning problems. |
| Approach: | They propose a critic-refiner framework that localizes and debugs visual programs by tracking execution step by step. |
| Outcome: | The proposed framework detects and corrects program errors leveraging detailed execution feedback, improving interpretability and accuracy. |
RePair: Automated Program Repair with Process-based Feedback (2024.findings-acl)
Copied to clipboard
| Challenge: | Commercial-scale language models (LMs) have taken APR to unprecedented levels, but they are limited by parameters and humans interact with them through explicit prompts. |
| Approach: | They propose a method that utilizes process supervision to improve program repair by allowing users to input feedback from compilers and test cases. |
| Outcome: | The proposed method outperforms large outcome-based generation methods and is inspired by strategies used in programming competitions. |
Do You See Me : A Multidimensional Benchmark for Evaluating Visual Perception in Multimodal LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) show reasoning promise, yet their visual perception is a bottleneck. |
| Approach: | They propose a visual perception benchmark to test the visual perception of MLLMs. |
| Outcome: | The proposed benchmark examines MLLMs' visual perception abilities with 1758 images and 2612 questions. |
CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios (2025.emnlp-main)
Copied to clipboard
| Challenge: | a number of tools are used to perform complex tasks, but the tool utilization process can cause errors. |
| Approach: | They propose a critique evaluation benchmark for tool learning that analyzes function-calling errors on tool evaluation benchmarks. |
| Outcome: | The proposed critique evaluation benchmark holds diverse tool-use errors with varying complexities, which better reflects real-world scenarios. |
Automatic Reviewers Fail to Detect Faulty Reasoning in Research Papers: A New Counterfactual Evaluation Framework (2026.tacl-1)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used as fully automatic review generators (ARGs). |
| Approach: | They propose a fully automated counterfactual evaluation framework that isolates and tests a core review skill that underpins high-quality peer review: detecting faulty research logic. |
| Outcome: | The proposed framework isolates and tests a range of ARG approaches and shows that flaws in research logic have no significant effect on their output reviews. |
MMRefine: Unveiling the Obstacles to Robust Refinement in Multimodal Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances have enabled MLLMs to tackle complex challenges such as mathematical reasoning and multimodal understanding. |
| Approach: | They propose a multimodal refinement benchmark to evaluate the refinement capabilities of Multimodal Large Language Models (MLLMs) the benchmark categorizes errors into six error types to highlight areas for improvement in effective reasoning enhancement. |
| Outcome: | The proposed framework evaluates the refinement capabilities of multimodal large language models across six scenarios. |
MMErroR: A Benchmark for Erroneous Reasoning in Vision-Language Models (2026.acl-long)
Copied to clipboard
Yang Shi, Yifeng Xie, Minzhe Guo, Liangsi Lu, Mingxuan Huang, Jingchao Wang, Zhihong Zhu, Boyan Xu, Zhiqi Huang
| Challenge: | Recent advances in vision-language models have improved performance in multi-modal learning. |
| Approach: | They propose a multi-modal benchmark that embeds a single coherent reasoning error in 1997 samples. |
| Outcome: | The proposed benchmark is based on a set of 1997 samples embedding a single coherent reasoning error. |
TURTLEAI: Benchmarking Multimodal Models for Visual Programming in Turtle Graphics (2026.findings-acl)
Copied to clipboard
| Challenge: | Vision-language models have been explored for visual programming, but performance is unclear . most prior work focuses on visual programming for productivity . |
| Approach: | They propose a visual programming benchmark that uses visual programming to evaluate VLMs. |
| Outcome: | The proposed model improves on GPT-5, GPT-4o, and Qwen2-VL-72B on real-world tasks by 20% . the proposed model is based on 823 visual programming tasks in the Turtle Graphics domain . |