IFEVAL

Papers with IFEval

Efficient Model Development through Fine-tuning Transfer (2025.emnlp-main)

Copied to clipboard

Pin-Jie Lin, Rishab Balasubramanian, Fengyuan Liu, Nikhil Kandpal, Tu Vu

Challenge:	Modern large language models face a major bottleneck: each new version of a pre-trained model requires expensive and repetitive alignment.
Approach:	They propose a method that transfers fine-tuning updates across model versions . they extract the diff vector, which is the difference in parameters induced by fine-uning, from a source model and apply it to the base of a different target model.
Outcome:	The proposed method reduces training costs while maintaining model performance.

M-IFEval: Multilingual Instruction-Following Evaluation (2025.findings-naacl)

Copied to clipboard

Antoine Dussolle, A. Cardeña, Shota Sato, Peter Devine

Multilinguality

Challenge:	Instruction following is a core capability of Large language models (LLMs), making evaluating this capability essential to understanding these models.
Approach:	They propose a multilingual instruction following evaluation benchmark that expands to other languages . they propose to use both general and language-specific instructions to evaluate LLMs .
Outcome:	The proposed benchmark is extended to French, Japanese, and Spanish . it shows that performance across languages and instruction types can vary widely .

Revisiting the Reliability of Language Models in Instruction-Following (2026.acl-long)

Copied to clipboard

Jianshuo Dong, Yutong Zhang, Liu Yan, Zhenyu Zhong, Tao Wei, Chao Zhang, Han Qiu

Semantic Text Processing

Challenge:	Several benchmarks have been proposed to measure instruction-following accuracy, but these scores do not translate to reliable services in real-world use.
Approach:	They propose a new metric reliable@k and develop an automated pipeline to generate cousin prompts.
Outcome:	The proposed model can be instantiated with cousin prompts and generates high-quality cousin prompt data.

LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints (2024.findings-emnlp)

Copied to clipboard

Thomas Palmeira Ferraz, Kartik Mehta, Yu-Hsiang Lin, Haw-Shiuan Chang, Shereen Oraby, Sijia Liu, Vivek Subramanian, Tagyoung Chung, Mohit Bansal, Nanyun Peng

Language Models

Challenge:	Recent studies have shown that LLMs struggle with instructions containing multiple constraints.
Approach:	They propose a self-correction pipeline that decomposes the original instruction into a list of constraints and uses a Critic model to decide when and where the LLM’s response needs refinement.
Outcome:	The proposed model outperforms GPT-4 on RealInstruct and IFEval even with weak feedback.

The Price of Format: Diversity Collapse in LLMs (2025.findings-emnlp)

Copied to clipboard

Longfei Yun, Chenyang An, Zilong Wang, Letian Peng, Jingbo Shang

Language Models

Challenge:	Instruction-tuned large language models employ structured templates to enforce format consistency during inference.
Approach:	They fine-tune instruction-tuning large language models with structured templates and evaluate their results across three axes: downstream task performance, alignment behavior, and output diversity.
Outcome:	The proposed model generates semantically similar outputs even under high temperature sampling and structural tokens in templates significantly constrain the model’s output space.

PaTaRM: Bridging Pairwise and Pointwise Signals via Preference-Aware Task-Adaptive Reward Modeling (2026.acl-long)

Copied to clipboard

Ai Jian, Jingqing Ruan, Xing Ma, Dailin Li, Weipeng Zhang, Ke Zeng, Xunliang Cai

Challenge:	Existing reward models lack generative and reasoning capabilities, resulting in poor performance.
Approach:	They propose a reward-aware task-adaptive reward model that enables pointwise training using readily available pairwise data via a novel Preference-Aware Reward mechanism.
Outcome:	The proposed reward model achieves an average relative improvement of 8.7% over the base models on RewardBench and RMBench.

Marco-Bench-MIF: On Multilingual Instruction-Following Capability of Large Language (2025.acl-long)

Copied to clipboard

Bo Zeng, Chenyang Lyu, Sinuo Liu, Mingyan Zeng, Minghao Wu, Xuanfan Ni, Tianqi Shi, Yu Zhao, Yefeng Liu, Chenyu Zhu, Ruizhe Li, Jiahui Geng, Qing Li, Yu Tong, Longyue Wang, Weihua Luo, Kaifu Zhang

Multilinguality

Challenge:	Existing datasets for instruction-following are monolingual and centered on English . existing data are unable to capture linguistic and cultural subtle differences .
Approach:	They propose an extension of IFEval to a localized multilingual version called Marco-Bench-MIF . their benchmark addresses linguistic constraints and cultural references via translation and verification .
Outcome:	The proposed extension of IFEval to a localized multilingual version covers 30 languages with varying levels of localization.

Hallucinations as Orthogonal Noise: Inference-Time Manifold Alignment via Dynamic Contextual Orthogonalization (2026.findings-acl)

Copied to clipboard

Mingkuan Zhao, Wentao Hu, Tianchen Huang, Yuheng Min, Suquan Chen, Yide Gao, Yanbo Zhai, Shuangyong Song, Xuelong Li

Reasoning

Challenge:	Hallucinations in Large Language Models persist in critical domains where generated content diverges from contextual facts or logical constraints.
Approach:	They propose to generate hallucinations as orthogonal noise relative to the semantic manifold of the residual stream.
Outcome:	The proposed method achieves superior contextual faithfulness compared to state-of-the-art methods.

What Tokens Truly Matter? The Logit Conflation Problem in LLM Sampling (2026.findings-acl)

Copied to clipboard

Pinlong Zhao, Huijun Tang, Pengfei Jiao, Mengyang Li

Language Models

Challenge:	Existing methods for large language models filter tokens based on logit magnitudes or derived statistics, under the implicit assumption that high-logit tokens are desirable.
Approach:	They propose to isolate the Logit Conflation Problem by using attention-weighted attribution to extract prompt-relevance from token logits.
Outcome:	The proposed method improves on LLaMA-3 and is training-free and low latency.

The Pitfalls of KV Cache Compression (2026.acl-long)

Copied to clipboard

Alex Chen, Renato Geh, Aditya Grover, Guy Van Den Broeck, Daniel Mingyi Israel

Low-Resource NLP

Challenge:	Recent literature has shown minimal degradation of KV cache in multi-instruction prompts . authors show that certain instructions degrade much more rapidly with compression .
Approach:	They propose to change KV cache eviction policies to reduce the impact of KV evict bias . they propose to use a 'simple' evviction policy to reduce ejection bias if the LLM is a multi-instruction model .
Outcome:	The proposed methods show that certain instructions degrade much faster with compression, causing them to be ignored by the LLM.

MDP-GRPO: Stabilized Group Relative Policy Optimization for Multi-Constraint Instruction Following (2026.acl-long)

Copied to clipboard

Mohammad Mahdi Salmani-Zarchi, Zahra Rahimi, Heshaam Faili, Mohammad Javad Dousti

Challenge:	Large language models (LLMs) can follow many natural-language instructions, yet they remain brittle when a request bundles multiple explicit constraints, such as asking the LLM to respond in a particular structure with an exact ending phrase.
Approach:	They propose a method which stabilizes learning through multi-temperature sampling to increase reward dispersion, dual-anchor advantages to restore gradients in homogeneous groups, prospect-theoretic shaping to bound updates and penalize violations based on Kahneman Tversky’s theory and asymmetric KL regularization.
Outcome:	The proposed method outperforms standard GRPO on FollowBench, IFEval, and a curated multi-constraint dataset, improving strict constraint satisfaction by up to 5.0% on Llama-3.2-3B.