Analyzing LLM Instruction Optimization for Tabular Fact Verification (2026.findings-eacl)
Copied to clipboard
Xiaotang Du, Giwon Hong, Wai-Chung Kwan, Rohit Saxena, Ivan Titov, Pasquale Minervini, Emily Allaway
| Challenge: | evaluating instruction optimization for tabular fact verification is a key challenge for reliable NLP systems. |
| Approach: | They compare instruction optimization for tabular fact verification with a framework based on DSPy . they find that instruction optimization consistently improves verification accuracy . |
| Outcome: | The proposed method improves verification accuracy across four benchmarks and three model families. |
Similar Papers
Program-Aided Reasoners (Better) Know What They Know (2024.naacl-long)
Copied to clipboard
| Challenge: | Prior work shows that program-aided reasoning improves accuracy but also requires reasoners to "know what they know". |
| Approach: | They compare the calibration of program-aided language models (PAL) and text-based Chain-of-thought (COT) prompting techniques over 5 datasets and 2 model types . |
| Outcome: | The proposed methods improve accuracy and calibrate the models over 5 datasets and 2 model types. |
FIPO: Free-form Instruction-oriented Prompt Optimization with Preference Dataset and Modular Fine-tuning Schema (2025.coling-main)
Copied to clipboard
| Challenge: | naive prompts can enhance the task performance of large language models, but they are resource-intensive. |
| Approach: | They propose an automatic prompt optimization method that refines naive prompts according to task outputs from in-box testing models. |
| Outcome: | The proposed method is based on a large-scale dataset and performed fairly across multiple models. |
ADO: Automatic Data Optimization for Inputs in LLM Prompts (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent research has focused on refining instruction components and augmenting input data with in-context examples, but this study explores the potential benefits of optimizing the input data itself. |
| Approach: | They propose a content engineering and structural reformulation strategy to optimize input data within prompts to improve performance of Large Language Models. |
| Outcome: | The proposed approach improves performance of Large Language Models (LLMs) in various tasks, offering a promising avenue for future research in prompt engineering. |
Black-Box Prompt Optimization: Aligning Large Language Models without Model Training (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models are often not well aligned with human intents, which requires additional training. |
| Approach: | They propose to use Black-Box Prompt Optimization (BPO) to perform alignments on large language models that are not well aligned with human intents. |
| Outcome: | The proposed model outperforms existing models and is model-agnostic. |
DICE: Structured Reasoning in LLMs through SLM-Guided Chain-of-Thought Correction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) often prioritize reasoning over adherence to detailed instructions due to high computational costs and limited parameter access. |
| Approach: | They propose a lightweight framework that guides small language models to refine LLMs’ outputs through chain-of-thought correction. |
| Outcome: | The proposed framework improves the average format accuracy and content correctness of LLM outputs by 35.4% and 29.4%, respectively, achieving state-of-the-art (SOTA) performance over other competitive baselines. |
Reasoning Up the Instruction Ladder for Controllable Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Current models struggle to balance competing directives, causing conflicting instructions. |
| Approach: | They propose to reframe instruction hierarchy resolution as a reasoning task . they use a training dataset to enable this capability by transferring general reasoning capabilities to instruction prioritization . |
| Outcome: | The proposed method improves on safety-critical scenarios beyond the training distribution and jailbreaks. |
ReasonIF: Large Reasoning Models Fail to Follow Instructions During Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior studies assess instruction adherence in the model’s main responses, but it is also critical for large reasoning models to follow user instructions throughout their reasoning process. |
| Approach: | They propose a systematic benchmark for assessing reasoning instruction following to assess the model's adherence to instructions. |
| Outcome: | The proposed benchmark reduces the risk of undesirable shortcuts, hallucinations, or reward hacking within reasoning traces. |
Heuristic-based Search Algorithm in Automatic Instruction-focused Prompt Optimization: A Survey (2025.findings-acl)
Copied to clipboard
Wendi Cui, Jiaxin Zhang, Zhuohang Li, Hao Sun, Damien Lopez, Kamalika Das, Bradley A. Malin, Sricharan Kumar
| Challenge: | Recent advances in Large Language Models (LLMs) have led to remarkable achievements across a variety of NLP tasks. |
| Approach: | They propose a taxonomy of automatic prompt optimization methods that explore and improve prompts with minimal human oversight. |
| Outcome: | The proposed methods can explore and improve prompts with minimal human oversight. |
Enhancing Reasoning Abilities of Small LLMs with Cognitive Alignment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to distill chain-of-thought (CoT) results from large language reasoning models (LRMs) to small models are ineffective and require substantial amount of annotated data. |
| Approach: | They propose a Critique-Rethink-Verify system for training small language reasoning models that can be critiquized according to the cognitive capabilities of smaller models. |
| Outcome: | The proposed system outperforms other methods on challenging reasoning benchmarks. |
Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have a number of shortcomings, including lack of factual correctness. |
| Approach: | They propose a framework to increase prediction factuality by post-editing reasoning chains . they propose to use large language models to generate interpretable reasoning chains. |
| Outcome: | The proposed framework leads to accuracy improvements in open-domain question-answering tasks. |