Visual Self-Refinement for Autoregressive Models

Challenge:	Autoregressive models excel in sequential modeling but the spatial nature of visual signals conflicts with the sequential dependencies of next-token prediction, leading to suboptimal results.
Approach:	They propose a plug-and-play refinement module to enhance the spatial correspondence modeling within the generated visual sequence.
Outcome:	The proposed module enhances vision-language modeling under a shared sequential prediction framework.

Similar Papers

Overview

Challenge

Approach

Outcome

Mitigating Sequential Dependencies: A Survey of Algorithms and Systems for Generation-Refinement Frameworks in Autoregressive Models (2025.findings-emnlp)

Copied to clipboard

Yunhai Hu, Zining Liu, Zhenyuan Dong, Tianfan Peng, Bradley McDanel, Sai Qian Zhang

Challenge:	Sequential dependencies present a fundamental bottleneck in deploying large-scale autoregressive models .
Approach:	They analyze methods based on generation strategies and refinement mechanisms . they examine deployment strategies across computing environments and explore applications spanning text, images, and speech generation.
Outcome:	The proposed frameworks can be used to improve the quality of autoregressive models.

Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement (D18-1)

Copied to clipboard

Jason Lee, Elman Mansimov, Kyunghyun Cho

Text Generation

Challenge:	Despite its success, neural autoregressive modeling has its weakness in decoding, i.e., finding the most likely sequence.
Approach:	They propose a conditional non-autoregressive neural sequence model based on iterative refinement based upon latent variable models and conditional denoising autoencoders.
Outcome:	The proposed model significantly speeds up decoding while maintaining the generation quality comparable to the autoregressive counterpart.

SCMAPR: Self-Correcting Multi-Agent Prompt Refinement for Complex-Scenario Text-to-Video Generation (2026.findings-acl)

Copied to clipboard

Chengyi Yang, Pengzhen Li, Jiayin Qi, Aimin Zhou, Ji Wu, Ji Liu

Language Models

Challenge:	Text-to-Video (T2V) generation is a challenge under complex scenarios.
Approach:	They propose a scenario-aware and self-correcting multi-agent prompt refinement framework for T2V prompting.
Outcome:	The proposed framework improves text-to-video alignment and overall generation quality under complex scenarios.

Self-Correcting Text-to-Video Generation with Misalignment Detection and Localized Refinement (2026.findings-acl)

Copied to clipboard

Daeun Lee, Jaehong Yoon, Jaemin Cho, Mohit Bansal

Multimodality

Challenge:	Recent text-to-video models struggle to faith-fully follow text prompts, authors say . authors propose a new refinement framework that detects fine-grained misalignments .
Approach:	They propose a video refinement framework that detects fine-grained misalignments . they propose preserving regions that should be preserved rather than regenerated .
Outcome:	The proposed framework detects fine-grained misalignments and performs targeted corrections . it preserves correctly generated entities, segments regions across frames, and regenerates problematic regions .

Hybrid Autoregressive-Diffusion Model for Real-Time Sign Language Production (2026.acl-long)

Copied to clipboard

Maoxiao Ye, Xinfeng Ye, Sathiamoorthy Manoharan

Challenge:	Existing approaches to sign language production use autoregressive or diffusion models that generate one-by-one output tokens but suffer from exposure bias during inference.
Approach:	They propose a hybrid autoregressive-diffusion model that combines iterative refinement and sequential dependency modeling for Sign Language production.
Outcome:	The proposed model improves sign language production quality and real-time efficiency on PHOENIX14T and How2Sign.

Autoregressive Semantic Visual Reconstruction Helps VLMs Understand Better (2026.findings-acl)

Copied to clipboard

Dianyi Wang, Wei Song, Yikun Wang, Siyuan Wang, Kaicheng Yu, Zhongyu Wei, Jiaqi Wang

Multimodality

Challenge:	Typical large vision-language models emphasize vision-to-language alignment while overlooking fine-grained visual information.
Approach:	They introduce autoregressive semantic visual reconstruction (ASVR) that enables joint learning of visual and textual modalities within a unified autoregression framework.
Outcome:	The proposed model improves baselines and multimodal understanding benchmarks by 2-3%.

Non-Autoregressive Models for Fast Sequence Generation (2022.emnlp-tutorials)

Copied to clipboard

Yang Feng, Chenze Shao

Multimodality

Challenge:	Autoregressive (AR) models can only generate target sequence word-by-word due to the AR mechanism and suffer from slow inference.
Approach:	This tutorial provides an introduction to non-autoregressive sequence generation.
Outcome:	This tutorial explains how to generate non-autoregressive sequence generation models.

Non-Autoregressive Text Generation with Pre-trained Language Models (2021.eacl-main)

Copied to clipboard

Yixuan Su, Deng Cai, Yan Wang, David Vandyke, Simon Baker, Piji Li, Nigel Collier

Semantic Text Processing

Challenge:	Autoregressive generation models generate tokens in a left-to-right, token-by-token fashion, resulting in lag in inference.
Approach:	They propose to use BERT as the backbone of a non-autoregressive generation model for greatly improved performance.
Outcome:	The proposed model outperforms existing non-autoregressive models and achieves competitive performance with many strong autoregressive model.

Self-AMPLIFY: Improving Small Language Models with Self Post Hoc Explanations (2024.emnlp-main)

Copied to clipboard

Milan Bhan, Jean-Noël Vittaut, Nicolas Chesneau, Marie-Jeanne Lesot

Responsible & Trustworthy NLP

Challenge:	Autoregressive Large Language Models (LLMs) have demonstrated "emergent abilities" such as in-context learning, instruction following and reasoning.
Approach:	They propose a method that generates rationales from post hoc explanation methods applied to small language models to improve their own performance.
Outcome:	The proposed method improves on four SLMs and five datasets with strong reasoning abilities.

Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation (2021.findings-acl)

Copied to clipboard

Peng Wang, Junyang Lin, An Yang, Chang Zhou, Yichang Zhang, Jingren Zhou, Hongxia Yang

Structured Data in NLP

Challenge:	Existing methods for table-to-text generation suffer from poor faithfulness and low coverage.
Approach:	They propose a method that combines Autoregressive and Non-Autoregressive generation to generate a table-to-text from a key-value table using a skeleton and an edit-based non-autoregressively generation model.
Outcome:	The proposed method outperforms the existing methods on WikiPerson and WikiBio datasets on coverage and faithfulness.