Papers with multi-step
Enhancing Ethical Explanations of Large Language Models through Iterative Symbolic Refinement (2024.eacl-long)
Copied to clipboard
| Challenge: | Recent studies have focused on the application and evaluation of Large Language Models (LLMs) but LLMs are still prone to factual errors and inconsistencies in their explanations, offering limited control and interpretability for inference in complex domains. |
| Approach: | They propose an abductive-deductive framework that integrates Large Language Models with an external backward-chaining solver to refine step-wise natural language explanations. |
| Outcome: | The proposed framework improves explanations generated via in-context learning methods and Chain-of-Thought (CoT) on ethical NLI tasks while producing formal proofs describing and supporting models’ reasoning. |
From Coarse to Fine: Self-Adaptive Hierarchical Planning for LLM Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing plans for large language model-based agents are limited by their granularity and lack flexibility. |
| Approach: | They propose a self-adaptive hierarchical planning mechanism that mimics human planning strategies and generates self-adapted hierarchic plans tailored to the varying difficulty levels of different tasks. |
| Outcome: | The proposed method significantly improves task execution success rates while mitigating overthinking at the planning level, providing a flexible and efficient solution for multi-step complex decision-making tasks. |
ASR-EC Benchmark: Evaluating Large Language Models on Chinese ASR Error Correction (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Automatic Speech Recognition (ASR) systems have a substantial number of erroneous recognition due to environmental noise, ambiguity, etc. |
| Approach: | They use a benchmark dataset to analyze ASR errors in the Chinese language . they then apply large language models to correct ASR error correction . |
| Outcome: | The proposed method is based on a dataset of ASR errors in the Chinese language . it shows prompting is not effective for ASR error correction . |
Cascaded Mutual Modulation for Visual Reasoning (D18-1)
Copied to clipboard
| Challenge: | Visual reasoning is a multi-step and compositional problem that requires intensive text-vision interactions. |
| Approach: | They propose a visual reasoning model that uses a feature-wise linear modulation technique to enable textual/visual pipelines to mutually control each other. |
| Outcome: | The proposed model outperforms existing models on visual reasoning benchmarks CLEVR and NLVR . it can generate a textual answer to a visual question answering problem with images . |
Test-Time Scaling of Reasoning Models for Machine Translation (2026.eacl-long)
Copied to clipboard
| Challenge: | Using TTS, Reasoning Models (RMs) are able to perform tasks such as math and coding with limited results. |
| Approach: | They evaluate 12 Reasoning Models across a diverse suite of MT benchmarks, examining three scenarios: direct translation, forced-reasoning extrapolation, and post-editing. |
| Outcome: | The proposed approach improves translation quality on three domains, with inconsistent results for general-purpose RMs and performance plateauing. |
SMART-Editor: A Multi-Agent Framework for Human-Like Design Editing with Structural Integrity (2026.findings-eacl)
Copied to clipboard
| Challenge: | SMART-EDITOR is a framework for compositional layout and content editing for structured visual domains. |
| Approach: | They introduce a framework for compositional editing for structured images like posters or websites . SMART-EDITOR maintains global coherence through two complementary strategies . |
| Outcome: | The proposed framework maintains global coherence through two complementary strategies. |
PRompt Optimization in Multi-Step Tasks (PROMST): Integrating Human Feedback and Heuristic-based Sampling (2024.emnlp-main)
Copied to clipboard
| Challenge: | Prompt optimization aims to find the best prompt to a large language model (LLM) for a given task. |
| Approach: | They propose a method to optimize prompts for LLM-driven multi-step tasks using a human-designed feedback rule. |
| Outcome: | The proposed method outperforms human-engineered prompts and several other prompt optimization methods on 11 representative multi-step tasks. |
SALT: Step-level Advantage Assignment for Long-horizon Agents via Trajectory Graph (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities, but their application to complex, multi-step, and long-horizon tasks remains challenging. |
| Approach: | They propose a framework that provides a finer-grained advantage assignment derived solely from outcome rewards. |
| Outcome: | The proposed framework provides a finer-grained advantage assignment, derived solely from outcome rewards. |
When natural language is not enough: The limits of in-context learning demonstrations in multilingual reasoning (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have demonstrated the effectiveness of reasoning methods in eliciting multi-step reasoned answers from Large Language Models (LLMs) by leveraging in-context demonstrations. |
| Approach: | They investigate how well CoT and PAL perform across languages for arithmetic and symbolic reasoning tasks. |
| Outcome: | The proposed methods perform well in monolingual contexts, primarily in English, but have been limited in other languages. |
TutorialVQA: Question Answering Dataset for Tutorial Videos (2020.lrec-1)
Copied to clipboard
| Challenge: | a new question answering task on instructional videos is needed due to their verbose nature . factoid questions are only a small part of what people actually want to ask on video contents . |
| Approach: | They propose a question answering task on instructional videos based on video transcripts . they use a dataset consisting of 6,000 manually collected triples of (video, question, answer span) |
| Outcome: | The proposed task focuses on screencast tutorial videos pertaining to an image editing program. |
ToolSafety: A Comprehensive Dataset for Enhancing Safety in LLM-Based Agent Tool Invocations (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current models exhibit notable vulnerabilities in maintaining safety during multi-step tool interactions and in indirect harm scenarios. |
| Approach: | They propose a safety fine-tuning dataset to fine- tune LLMs into assistants . they propose to use synthesized trajectories and realistic, context-aware sample generation . |
| Outcome: | The proposed model maintains safety in multi-step and indirect harm scenarios with little impact on helpfulness. |
MLAlgo-Bench: Can Machines Implement Machine Learning Algorithms? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, the top-performing models achieve a 48.8% task completion rate on realizing machine learning algorithms . |
| Approach: | They propose a benchmark to test machine learning's ability to generate ML code for humans . they propose an automatic evaluation framework with metrics such as task pass rate and time overhead . |
| Outcome: | The proposed benchmark is unique in its focus on interpreting complex human instructions and producing multi-step, high-complexity code. |
BackdoorAgent: A Unified Framework for Backdoor Attacks on LLM-based Agents (2026.findings-acl)
Copied to clipboard
Yunhao Feng, Yige Li, Yutao Wu, Yingshui Tan, Yanming Guo, Yifan Ding, Kun Zhai, Xingjun Ma, Yu-Gang Jiang
| Challenge: | Large language model (LLM) agents execute tasks through multi-step workflows that combine planning, memory, and tool use. |
| Approach: | They propose a modular framework that provides a unified view of backdoor threats in LLM agents. |
| Outcome: | The proposed framework provides a unified, agent-centric view of backdoor threats in LLM agents. |
From Capabilities to Performance: Evaluating Key Functional Properties of LLM Architectures in Penetration Testing (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been explored for automating or enhancing penetration testing tasks, but their effectiveness and reliability remain open questions. |
| Approach: | They evaluate multiple LLM-based agents across realistic penetration testing scenarios . they also examine impact of core functional capabilities on agent success . |
| Outcome: | The proposed models improve agent performance in multi-step and real-time penetration testing scenarios. |
SeqPO-SiMT: Sequential Policy Optimization for Simultaneous Machine Translation (2025.findings-acl)
Copied to clipboard
| Challenge: | SeqPO-SiMT is a new policy optimization framework for simultaneous machine translation that combines a tailored reward with a single step task. |
| Approach: | They propose a new policy optimization framework that defines the simultaneous machine translation task as a sequential decision making problem with a tailored reward. |
| Outcome: | The proposed framework outperforms the supervised fine-tuning model by 1.13 points while reducing the Average Lagging by 6.17 in the NEWSTEST2021 En Zh dataset. |
Context-attended Adversarial Reinforcement Learning for Robust Multi-step Retrieval Augmented Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to multi-step retrieval-augmented generation are susceptible to retrieval noise and fabricated documents in real-world scenarios. |
| Approach: | They propose a framework for multi-step retrieval-augmented generation that incorporates external knowledge into a retriever to generate responses from adversarial samples. |
| Outcome: | The proposed framework improves performance in multiple noisy scenarios and can be used to improve multi-step retrieval-augmented generation. |
GlimpRouter: Efficient Collaborative Inference by Glimpsing One Token of Thoughts (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing routing strategies rely on local token probabilities or post-hoc verification, introducing significant inference overhead. |
| Approach: | They propose a step-wise collaboration framework that generates only the first token of each reasoning step and routes it to a larger model only when initial token entropy exceeds a threshold. |
| Outcome: | The proposed approach reduces inference latency while preserving accuracy. |
C²RBench: A Chinese Complex Reasoning Benchmark for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks often fail to capture complex multi-step reasoning demands inherent in real-world scenarios. |
| Approach: | They propose a benchmark to evaluate multi-step, multimodal advanced reasoning of large language models. |
| Outcome: | The proposed benchmark exceeds existing benchmarks in cognitive complexity and accuracy by over 90% . it features 1,115 carefully curated Chinese tasks organized into eight domain-specific subsets . evaluations of 20 LLMs and 24 multimodal large language models reveal critical performance gaps . |
TurnBench-MS: A Benchmark for Evaluating Multi-Turn, Multi-Step Reasoning in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks focus on single-turn or single-step tasks, failing to capture iterative reasoning in real-world settings. |
| Approach: | They propose a benchmark that evaluates multi-turn, multi-step reasoning through an interactive code-breaking task inspired by the "Turing Machine Board Game" the best model achieves 84% accuracy in Classic mode, but performance drops to 18% in Nightmare mode. |
| Outcome: | The new benchmark evaluates multi-turn, multi-step reasoning through an interactive code-breaking task inspired by the "Turing Machine Board Game" the best model achieves 84% accuracy in Classic mode, but performance drops to 18% in Nightmare mode. |
Thinking Like a Botanist: Challenging Multimodal Language Models with Intent Driven Chain-of-Inquiry (2026.findings-acl)
Copied to clipboard
Syed Nazmus Sakib, Nafiul Haque, Shahrear Bin Amin, Hasan Muhammad Abdullah, Md Mehedi Hasan, Mohammad Zabed Hossain, Shifat E. Arman
| Challenge: | Visual question-based reasoning is a key component of vision-language models. |
| Approach: | They propose a framework for visual question-answering that integrates visual intent with visual severity to improve diagnostic accuracy. |
| Outcome: | The proposed framework improves diagnostic correctness, reduces hallucination, and increases reasoning efficiency. |
Chain-of-Thought as a Lens: Evaluating Structured Reasoning Alignment between Human Preferences and Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Empirical results show misalignment at greater reasoning depths is driven mainly by alignment errors such as thematic shift and redundant reasoning. |
| Approach: | They propose a method to quantitatively assess the alignment between multi-step, structured reasoning in large language models and human preferences by constructing semantic-entropy-based matrices over intermediate steps and measuring their divergence. |
| Outcome: | The proposed method shows that it is consistent with previous studies and can be used as a diagnostic signal. |
R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Qingfei Zhao, Ruobing Wang, Dingling Xu, Daren Zha, Ma Bowen, Zhichun Wang, Shijie Jia, Limin Liu, Xin Wang
| Challenge: | Large language models (LLMs) have demonstrated impressive capabilities in multi-step and long-chain reasoning, but extending their reasoning capabilities to encompass deep interactions with search remains a non-trivial challenge. |
| Approach: | They propose a framework for Reasoning–Search integration that integrates multi-reward signals to optimize the reasoning–search interaction trajectories. |
| Outcome: | Experiments on seven datasets show that R-Search significantly outperforms mainstream RAG baselines. |
DiffCoT: Diffusion-styled Chain-of-Thought Reasoning in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) reasoning improves multi-step mathematical problem solving in large language models but is vulnerable to exposure bias and error accumulation. |
| Approach: | They propose a diffusion-styled CoT framework that reformulates CoT reasoning as an iterative denoising process. |
| Outcome: | The proposed framework outperforms existing methods on three multi-step CoT reasoning benchmarks. |