PROBLEM-SOLVING

PolicyLLM: Towards Excellent Comprehension of Public Policy for Large Language Models (2026.findings-acl)

Challenge:	Existing diagnostic approaches rely on expensive expert annotations and ”LLM-as-a-judge” paradigms.
Approach:	They propose a framework for semantic failure attribution that identifies responsible agents and the originating error step.
Outcome:	The proposed framework outperforms baselines in step-level localization and validation.

Copied to clipboard

Han Bao, Penghao Zhang, Yue Huang, Zhengqing Yuan, Yanchi Ru, SU Rui, Yujun Zhou, Xiangqi Wang, Kehan Guo, Nitesh V Chawla, Yanfang Ye, Xiangliang Zhang

MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning (2025.findings-acl)

Challenge:	Large Language Models (LLMs) are increasingly integrated into real-world decision-making, but their ability to comprehend and reason about policy-related content remains underexplored.
Approach:	They propose a bilingual benchmark evaluating policy comprehension comprising 21K cases across a broad spectrum of policy areas.
Outcome:	The proposed model shows stronger performance on application-oriented policy tasks than on memorization or conceptual understanding, and yields the highest accuracy on structured reasoning tasks.

Copied to clipboard

Ke Wang, Junting Pan, Linda Wei, Aojun Zhou, Weikang Shi, Zimu Lu, Han Xiao, Yunqiao Yang, Houxing Ren, Mingjie Zhan, Hongsheng Li

LANS: A Layout-Aware Neural Solver for Plane Geometry Problem (2024.findings-acl)

Challenge:	Large Language Models (LMMs) struggle with simple tasks such as geometry, e.g., arithmetic, and reasoning.
Approach:	They propose to leverage code as supervision for cross-modal alignment . they propose to use FigCodifier and ImgCode-8.6M to synthesize novel mathematical figures .
Outcome:	The proposed model surpasses GPT-4o and Claude 3.5 Sonnet in the geometry problem-solving subset of MathVista, achieving improvements of 8.9% and 9.2%.

Copied to clipboard

Zhong-Zhi Li, Ming-Liang Zhang, Fei Yin, Cheng-Lin Liu

MIThinker: A Plug-and-Play Policy-Optimized Thinker For Motivational Interviewing Counseling (2026.findings-acl)

Challenge:	Existing neural solvers take GPS as vision-language task but lack layout awareness . Existing models are criticized for complex rules and poor adaptability .
Approach:	They propose a layout-aware neural solver called LANS that integrates two modules to solve GPS.
Outcome:	The proposed solver outperforms existing neural and symbolic solvers on two datasets.

Copied to clipboard

Yizhe Yang, Palakorn Achananuparp, Heyan Huang, Jing Jiang, Ee-Peng Lim

Meta-Reflection: A Feedback-Free Reflection Learning Framework (2025.acl-long)

Challenge:	Existing reasoning large language models (LLMs) generate responses without explicitly aligning thoughts with counseling techniques, limiting their effectiveness.
Approach:	They propose a lightweight thinking model that generates therapeutic thoughts to guide MI counseling agents in strategy selection and response generation.
Outcome:	The proposed model achieves theory-of-mind assessment comparable to state-of the-art systems with an order of magnitude less computation.

Copied to clipboard

Yaoke Wang, Yun Zhu, XintongBao XintongBao, Wenqiao Zhang, Suyang Dai, Kehan Chen, Wenqiang Li, Gang Huang, Siliang Tang, Yueting Zhuang

Information Retrieval

Challenge:	Existing approaches to improve large language models' ability to understand and reason are limited by external feedback.
Approach:	They propose a feedback-free reflection mechanism that requires only a single inference pass without external feedback.
Outcome:	The proposed method is based on an industrial e-commerce benchmark and public datasets.

NTSEBENCH: Cognitive Reasoning Benchmark for Vision Language Models (2025.findings-naacl)

Copied to clipboard

Pranshu Pandya, Vatsal Gupta, Agney S Talwarr, Tushar Kataria, Dan Roth, Vivek Gupta

TopoDIM: One-shot Topology Generation of Diverse Interaction Modes for Multi-Agent Systems (2026.findings-acl)

Challenge:	Recent advances in large language models have demonstrated their strong performance on IQ test questions, achieving high scores across many languages.
Approach:	They propose a dataset to evaluate cognitive multimodal reasoning and problem-solving skills of large models.
Outcome:	The proposed dataset contains 2,728 multiple-choice questions and 4,642 images spanning 26 categories.

Copied to clipboard

Rui Sun, Jie Ding, Chenghua Gong, Tianjun Gu, Yihang Jiang, Juyuan Zhang, Liming Pan, Linyuan Lü

Dialogue Systems & Conversational Agents

Challenge:	Existing communication topologies rely on spatio-temporal dialogues, which incur high latency and computation.
Approach:	They propose a framework for one-shot Topology generation with Diverse Interaction Modes that enables agents to construct heterogeneous communication without iterative coordination.
Outcome:	The proposed framework reduces total token consumption by 46.41% while improving average performance by 1.50% over state-of-the-art methods.

Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios (2024.findings-acl)

Copied to clipboard

Shijue Huang, Wanjun Zhong, Jianqiao Lu, Qi Zhu, Jiahui Gao, Weiwen Liu, Yutai Hou, Xingshan Zeng, Yasheng Wang, Lifeng Shang, Xin Jiang, Ruifeng Xu, Qun Liu

Hello Again! LLM-powered Personalized Agent for Long-term Dialogue (2025.naacl-long)

Challenge:	Existing benchmarks focus on simple synthesized queries that do not reflect real-world complexity, thereby offering limited perspectives in evaluating tool utilization.
Approach:	They propose a benchmark to evaluate LLMs’ ability in tool utilization within real-world scenarios.
Outcome:	The proposed benchmark improves LLMs’ ability in tool utilization within real-world scenarios and eliminates the restriction of pre-defined toolset.

Copied to clipboard

Hao Li, Chenghao Yang, An Zhang, Yang Deng, Xiang Wang, Tat-Seng Chua

Natural Language Interfaces

Challenge:	Existing dialogue systems focus on brief single-session interactions, neglecting real-world needs for long-term companionship and personalized interactions.
Approach:	They propose a model-agnostic framework for long-term dialogue agents . they use event summary and persona management to enable reasoning .
Outcome:	The proposed framework incorporates three independently tunable modules dedicated to event perception, persona extraction, and response generation.

Reasoning with Language Model Prompting: A Survey (2023.acl-long)

Copied to clipboard

Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, Huajun Chen

Rethinking Creativity Evaluation: A Critical Analysis of Existing Creativity Evaluations (2026.eacl-long)

Challenge:	Reasoning is an essential ability for complex problem-solving and can provide back-end support for various real-world applications.
Approach:	They present cutting-edge research on reasoning with language model prompting and provide systematic resources to help beginners.
Outcome:	The proposed approaches have not been systematically reviewed and analyzed.

Copied to clipboard

Li-Chun Lu, Miri Liu, Pin Chun Lu, Yufei Tian, Shao-Hua Sun, Nanyun Peng

Syntactic Text Processing

Challenge:	Creativity measures that distinguish creativity in one domain fail in others, and different metrics disagree on the same data points.
Approach:	They examine, analyze, and compare four representative creativity measures across the diverse creative domains, including creative writing, unconventional problem-solving, and research ideation.
Outcome:	The measures of creativity across creative domains are compared using a set of human-aligned examples and lack consistency across domains and metrics.

Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models (2023.emnlp-main)

Copied to clipboard

Daman Arora, Himanshu Singh, null Mausam

Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors (2024.emnlp-main)

Challenge:	The performance of large language models (LLMs) on existing reasoning benchmarks has significantly improved over the past decade.
Approach:	They propose a benchmark dataset for evaluating the problem solving abilities of large language models (LLMs) they curate 515 challenging problems from the highly competitive IIT JEE-Advanced exam.
Outcome:	The proposed model performs better on open-source and proprietary models than the current model, but with techniques like self-consistency, self-refinement and chain-of-thought prompting.

Copied to clipboard

Nico Daheim, Jakub Macina, Manu Kapur, Iryna Gurevych, Mrinmaya Sachan

CausalEval: Towards Better Causal Reasoning in Language Models (2025.naacl-long)

Challenge:	Existing models for dialog tutoring fail to detect student errors and tailor their feedback to them.
Approach:	They propose to build dialog tutoring models to scaffold students' problem-solving and verify student solutions by using automatic and human evaluation.
Outcome:	The proposed model improves the quality of the tutor response generation by detecting student errors and adjusting the feedback to the errors.

Copied to clipboard

Longxuan Yu, Delin Chen, Siheng Xiong, Qingyang Wu, Dawei Li, Zhikai Chen, Xiaoze Liu, Liangming Pan

Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correction (2024.findings-acl)

Challenge:	Large language models (LLMs) have been used for a variety of tasks, including problem-solving, decision-making, and understanding of the world.
Approach:	They propose a review of existing methods aimed at enhancing LMs for causal reasoning . they categorize existing methods as reasoning engines or as helpers providing knowledge or data to traditional methods .
Outcome:	The proposed methods perform better than existing methods on a range of tasks.

Copied to clipboard

Xiaoyuan Li, Wenjie Wang, Moxin Li, Junrong Guo, Yang Zhang, Fuli Feng

HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation Task (2025.findings-acl)

Challenge:	Existing evaluations focus on problem-solving from examiner perspective, overlooking a dual perspective of examiner regarding error identification and correction.
Approach:	They propose to use an annotated dataset to evaluate large language models from the examiner perspective and to use diverse prompts to evaluate eleven representative LLMs.
Outcome:	The proposed model outperforms all models while LLaMA-2-7B has comparable abilities to closed-source models GPT-3.5 and Gemini Pro.

Copied to clipboard

Zhaojian Yu, Yilun Zhao, Arman Cohan, Xiao-Ping Zhang

Programming Languages in NLP

Challenge:	Existing benchmarks for code generation tasks are inadequate, but performance declines on self-invoking tasks.
Approach:	They propose a general recipe for generating more challenging versions of existing benchmarks . they propose to use instruction-tuned models to evaluate LLMs on self-invoking code generation tasks .
Outcome:	The proposed model improves on humanEval and MBPP but on self-invoking code generation tasks.

Auto-Evolve: Enhancing Large Language Model’s Performance via Self-Reasoning Framework (2024.findings-emnlp)

Copied to clipboard

Krishna Aswani, Huilin Lu, Pranav Patankar, Priya Dhalwani, Xue Tan, Jayant Ganeshmohan, Simon Lacasse

ReSo: A Reward-driven Self-organizing LLM-based Multi-Agent System for Reasoning Tasks (2025.emnlp-main)

Challenge:	Recent advances in prompt engineering strategies rely on static seed reasoning modules to simulate human approach to problem-solving.
Approach:	They propose a framework that enables LLMs to self-create dynamic reasoning modules and downstream action plan.
Outcome:	The proposed framework outperforms existing prompting strategies on a BigBench-Hard dataset and improves performance by 2.8% over existing methods.

Copied to clipboard

Heng Zhou, Hejia Geng, Xiangyuan Xue, Li Kang, Yiran Qin, Zhiyong Wang, Zhenfei Yin, Lei Bai

How Far are LLMs from Being Our Digital Twins? A Benchmark for Persona-Based Behavior Chain Simulation (2025.findings-acl)

Challenge:	Multi-agent systems (MAS) are limited by poor flexibility and scalability, with underdeveloped optimization strategies.
Approach:	They propose a task graph generation and a reward-driven two-stage agent selection process to integrate multi-agent systems to improve their reasoning capabilities.
Outcome:	The proposed model outperforms existing methods on Math-MAS and SciBench-MAS SciBech, while other methods completely fail.

Copied to clipboard

Rui Li, Heming Xia, Xinfeng Yuan, Qingxiu Dong, Lei Sha, Wenjie Li, Zhifang Sui

Challenge:	Recent studies have focused on dialogue simulation while overlooking human behavior simulation, which is crucial for digital twins.
Approach:	They propose to integrate persona metadata into LLMs and use it to iteratively infer contextually appropriate behaviors within dynamic scenarios.
Outcome:	The proposed model is based on 15,846 distinct behaviors across 1,001 unique personas and incorporates persona metadata to iteratively infer appropriate behaviors within dynamic scenarios.

Think Earlier, Not Longer: Prompt Optimization via Reducing Unhealthy Exploration (2026.findings-acl)

Copied to clipboard

Ling-I Wu, Minyu Chen, Jingyang Li, Xi Chang, Guoqiang Li

Responsible & Trustworthy NLP

Challenge:	Existing approaches to improve reasoning performance ignore the presence of unhealthy exploration that increases token usage without contributing to effective problem-solving.
Approach:	They propose an entropy-dynamics-aware prompt optimization framework that trains a lightweight optimizer to generate concise clarifications.
Outcome:	The proposed framework reduces ambiguity-induced early-stage uncertainty while preserving the model's reasoning capabilities.

Enhanced Data Synthesis for LLM through Reasoning Structures Generated by Hierarchical GFlowNet (2025.findings-acl)

Copied to clipboard

Tianpeng Bu, Minying Zhang, Hongtao Duan, Shurui Li, Lulu Hu, Yu Li

Avoiding Knowledge Edit Skipping in Multi-hop Question Answering with Guided Decomposition (2025.findings-emnlp)

Challenge:	Existing methods to optimize instruction-response pairs lack a systematic design for the underlying reasoning structure.
Approach:	They propose a Reasoning Structure driven data Synthesis method that leverages a coarse-to-fine directed acyclic graph to construct reasoning structures efficiently.
Outcome:	The proposed method outperforms existing methods in 48.50%, 84.00%, 79.90% of the synthetic datasets trained on the proposed model.

Copied to clipboard

Yi Liu, Xiangrong Zhu, Xiangyu Liu, Wei Wei, Wei Hu

Natural Language Interfaces

Challenge:	Existing methods for knowledge editing fail to work in multi-hop question answering due to 'edit skipping' edit skipping occurs due to the mismatch between the granularity of LLMs in problem-solving and the facts in the edited memory.
Approach:	They propose a retrieval-augmented generation-based method that edits knowledge without modifying parameters without retraining LLMs.
Outcome:	The proposed method outperforms state-of-the-art methods for KE in multi-hop question answering.

MONAQ: Multi-Objective Neural Architecture Querying for Time-Series Analysis on Resource-Constrained Devices (2025.findings-emnlp)

Copied to clipboard

Patara Trirat, Jae-Gil Lee

Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication (2023.emnlp-main)

Challenge:	Recent efforts in hardware-aware neural architecture search (NAS) automate architecture discovery for specific platforms; however, none focus on general time-series analysis with edge deployment.
Approach:	They propose a framework that reformulates NAS into *Multi-Objective Neural Architecture Q*uerying tasks.
Outcome:	Experiments on 15 datasets show that the proposed framework outperforms both handcrafted models and NAS baselines while being more efficient.

Copied to clipboard

Zhangyue Yin, Qiushi Sun, Cheng Chang, Qipeng Guo, Junqi Dai, Xuanjing Huang, Xipeng Qiu

Mathador-LM: A Dynamic Benchmark for Mathematical Reasoning on Large Language Models (2024.emnlp-main)

Challenge:	Large Language Models (LLMs) have made significant strides in complex reasoning tasks, but their reasoning is often constrained by their intrinsic understanding, lacking external insights.
Approach:	They propose a framework that enables cross-model communication during problem-solving.
Outcome:	The proposed framework surpasses established baselines in complex reasoning tasks and is cost-effective.

Copied to clipboard

Eldar Kurtic, Amir Moeini, Dan Alistarh

Automated Creativity Evaluation of Language Models Across Open-Ended Tasks (2026.acl-long)

Challenge:	a new benchmark for evaluating the mathematical reasoning on large language models is being developed . popularity of reasoning benchmarks is leading to performance saturation and training set contamination.
Approach:	They introduce a benchmark for evaluating the mathematical reasoning on large language models . they find that models struggle with Mathador-LM, scoring lower than average 3rd graders .
Outcome:	The proposed benchmark improves performance on large language models . it also reduces test-set leakage into training data, a new study shows .

Copied to clipboard

Tan Min Sen, Zachary Choy Kit Chun, Syed Ali Redha Alsagoff, Nadya Yuki Wangsajaya, Banerjee Mohor, Swaagat Bikash Saikia, Alvin Chan

Semantic Text Processing

Challenge:	Existing methods for evaluating creativity are tightly coupled to specific tasks and limiting scalability and generality.
Approach:	They propose a domain-agnostic framework for quantifying LLM creativity across open-ended tasks.
Outcome:	The proposed framework captures key facets of creativity including novelty, diversity, and task fulfilment with over 60% improved efficiency.

SkyLLM: Cross-LLM-APIs Federation for Cost-effective Query Processing (2025.findings-acl)

Copied to clipboard

Heng Zhao, Yifei Zhu

Challenge:	Large language models (LLMs) have demonstrated exceptional capabilities across a wide range of tasks, from text generation to complex problem-solving.
Approach:	They propose a system which federates multiple LLM APIs and dynamically assigns a non-empty subset of these APIs to each query prior to inference.
Outcome:	The proposed system can match the most accurate LLM with the lowest cost while cutting costs by 67.8%.

The Role of Visual Modality in Multimodal Mathematical Reasoning: Challenges and Insights (2025.acl-long)

Copied to clipboard

Yufang Liu, Yao Du, Tao Ji, Jianing Wang, Yang Liu, Yuanbin Wu, Aimin Zhou, Mengdi Zhang, Xunliang Cai

PECC: Problem Extraction and Coding Challenges (2024.lrec-main)

Challenge:	Existing models that leverage visual information do not improve math reasoning performance . authors suggest that visual information is important for multimodal reasoning .
Approach:	They propose a dataset to require image reliance for problem-solving and challenge models with similar, yet distinct, images that change the correct answer.
Outcome:	The proposed model performance is unaffected by changes to or removal of images in the dataset.

Copied to clipboard

Patrick Haller, Jonas Golde, Alan Akbik

Information Extraction & Text Mining

Challenge:	Existing benchmarks evaluate tasks in isolation, yet the extent to which LLMs can understand prose-style tasks, identify the underlying problems, and then generate appropriate code solutions remains mostly unexplored.
Approach:	They propose a benchmark derived from Advent Of Code challenges and Project Euler, which requires LLMs to interpret narrative-embedded problems, extract requirements, and generate executable code.
Outcome:	The proposed benchmarks show that LLMs can understand prose-style tasks, identify underlying problems, and generate appropriate code solutions in a variety of tasks.

Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verification (2026.findings-acl)

Copied to clipboard

Yuxuan Wan, Tianqing Fang, Zaitang LI, Yintong Huo, Wenxuan Wang, Haitao Mi, Dong Yu, Michael R. Lyu

Challenge:	Recent advances in Deep Research Agents (DRAs) are transforming automated knowledge discovery and problem-solving.
Approach:	They propose an inference-time scaling of verification wherein an agent self-improves at test time by evaluating its generated answers.
Outcome:	The proposed model outperforms vanilla agent-as-judge and LLM judge baselines by 12%–48% in meta-evaluation F1 score.

Evolving Agentic Workflow Driven by Human-Agent Collaboration (2026.findings-acl)

Copied to clipboard

Yuxin Liu, Jinxuan Zhang, Yuezhang Peng, Hefeng Zhou, Xiangfeng Wang, Jiong Lou, Chentao Wu, Jie LI, Jingjing Qu, Chaochao Lu

Challenge:	Existing approaches to generate agentic workflows using large language models are limited by high manual design costs, inefficient agentic search, and poor dynamic adaptability to new tasks and human preferences.
Approach:	They propose an evolutionary framework for generating agentic workflows through human-agent collaboration using evolutionary algorithms that mutate and cross over their structures, prompts, and LLM backbones.
Outcome:	The proposed framework surpasses other automated baselines by 27.34% while achieving comparable performance to o1-preview at only one-fourth of the cost.

MathCanvas: Intrinsic Visual Chain-of-Thought for Multimodal Mathematical Reasoning (2026.acl-long)

Copied to clipboard

Weikang Shi, Aldrich Yu, Rongyao Fang, Houxing Ren, Ke Wang, Aojun Zhou, Changyao Tian, Xinyu Fu, Yuxuan Hu, Zimu Lu, Linjiang Huang, Si Liu, Rui Liu, Hongsheng Li

GeoPQA: Bridging the Visual Perception Gap in MLLMs for Geometric Reasoning (2025.findings-emnlp)

Challenge:	Existing approaches to visual chain-of-thought are limited by external tools or fail to generate high-fidelity diagrams.
Approach:	They propose a framework to enable large multimodal models with VCoT capabilities . they pre-train a model on a 15.2M-pair corpus and teach it how to leverage visual aids .
Outcome:	The proposed framework unlocks complex, human-like visual reasoning in large language models . it pre-trains the model on a 15.2M-pair corpus and fine-tunes it on MathCanvas-Instruct .

Copied to clipboard

Guizhen Chen, Weiwen Xu, Hao Zhang, Hou Pong Chan, Deli Zhao, Anh Tuan Luu, Yu Rong

Inventive Problem Solving with LLMs: A Benchmark for TRIZ Reasoning (2026.findings-acl)

Challenge:	Recent advances in reinforcement learning (RL) have enhanced the reasoning abilities of large language models, but the impact on multimodal LLMs is limited.
Approach:	They propose a two-stage RL framework that enhances visual perception and fosters reasoning capabilities.
Outcome:	The proposed framework improves geometric reasoning by 9.7% and problem-solving by 9.1% compared to direct reasoning training approach.

Copied to clipboard

Zhu Wang, Brian Uzzi