Generative Floor Plan Design with LLMs via Reinforcement Learning with Verifiable Rewards

Challenge:	Existing generative models focus on respecting the requested connectivity between rooms, but do not support generating floor plans that respect numerical constraints.
Approach:	They propose a text-based approach that fine-tunes a large language model on real plans and applies reinforcement learning with verifiable rewards to improve adherence to topological and numerical constraints.
Outcome:	The proposed model outperforms existing methods on Realism, Compatibility, Diversity metrics.

Similar Papers

Overview

Challenge

Approach

Outcome

FloorPlan-LLaMa: Aligning Architects’ Feedback and Domain Knowledge in Architectural Floor Plan Generation (2025.acl-long)

Copied to clipboard

Jun Yin, Pengyu Zeng, Haoyuan Sun, Yuqin Dai, Han Zheng, Miao Zhang, Yachao Zhang, Shuai Lu

Challenge:	Existing evaluation methods for floor plan generation rely on statistical metrics like FID, GED, and PSNR, which fail to evaluate using domain knowledge.
Approach:	They propose to use a first floor plan dataset to train a floor plan generation model based on a multi-dimensional preference score and a textual analysis to integrate architects’ professional expertise and preferences.
Outcome:	The proposed model outperforms baseline models in text-conditional and class-condition tasks and is more rational and aligns better with human preferences.

Crossing the Reward Bridge: Expanding Reinforcement Learning with Verifiable Rewards Across Diverse Domains (2026.acl-long)

Copied to clipboard

Yi Su, Dian Yu, Linfeng Song, Juntao Li, Haitao Mi, Zhaopeng Tu, Min Zhang, Dong Yu

Challenge:	Reinforcement learning with verifiable rewards (RLVR) has been effective on structured tasks, but its reliance on simple, rule-based verifiers creates a bottleneck.
Approach:	They propose a framework that uses a generative verifier to provide soft, probabilistic rewards.
Outcome:	The proposed framework outperforms existing models up to 10x their size and can be scalable and effective.

Tell2Design: A Dataset for Language-Guided Floor Plan Generation (2023.acl-long)

Copied to clipboard

Sicong Leng, Yang Zhou, Mohammed Haroon Dupty, Wee Sun Lee, Sam Joyce, Wei Lu

Multimodality

Challenge:	Recent studies have demonstrated impressive results in generating high-fidelity artistic images.
Approach:	They propose a Sequence-to-Sequence model that can serve as a strong baseline for future research.
Outcome:	The proposed model can be used as a baseline for future research and human evaluations are conducted on the generated samples and provided an analysis of human performance.

Teaching LLMs to Plan, Not Just Solve: Plan Learning Boosts LLMs Generalization in Reasoning Tasks (2025.findings-emnlp)

Copied to clipboard

Tianlong Wang, Junzhe Chen, Weibin Liao, Xueting Han, Jing Bai

Reasoning

Challenge:	Existing methods for reinforcement learning (RL) on self-generated data are limited in many domains.
Approach:	a new framework combines plan-based search with Step-level Advantage Preference Optimization to optimize plan learning.
Outcome:	The proposed framework improves in-domain performance and out-of-domain benchmarks.

Aligning Large Language Models via Fully Self-Synthetic Data (2026.acl-long)

Copied to clipboard

Shangjian Yin, Zhepei Wei, Xinyu Zhu, Wei-Lin Chen, Yu Meng

Language Models

Challenge:	Existing approaches to reinforcement learning from human feedback (RLHF) require expensive human-annotated datasets and proprietary models like GPT-4 to annotate preference pairs.
Approach:	They propose a self-synthetic framework for LLM alignment where all training data, including prompts (i.e., user queries), responses, and preferences, are generated by the model itself.
Outcome:	The proposed framework enhances the model’s chat capabilities on standard benchmarks like AlpacaEval 2.0 while maintaining strong performance on downstream objective tasks.

CARL: Constraint-Aware Reinforcement Learning for Planning with LLMs (2026.findings-acl)

Copied to clipboard

Qiuyi Qi, Jinjian Zhang, Mutian Bao, Tian Liang, Guocong Li, Dongnan Liu, Wei Zhou, Jie Liu, Ming Kong, Linjian Mo, Feng Zhang, Qiang Zhu

Reasoning

Challenge:	Existing approaches to constraint-aware planning fail to enhance the model’s intrinsic focus on constraints.
Approach:	They propose a constraint-aware reinforcement learning framework that encourages constraint focus and penalizes neglect of LLMs.
Outcome:	The proposed framework outperforms existing frameworks and state-of-the-art reasoning models in a number of real-world applications.

Verifying the Subjective: Structured Multilingual Rewards for Low-Resource Alignment (2026.findings-acl)

Copied to clipboard

Jiu Sha, Mengxiao Zhu

Responsible & Trustworthy NLP

Challenge:	Structured Multilingual Reward Modeling Framework extends Reinforcement Learning with Verifiable Rewards (RLVR) to subjective and open-ended tasks.
Approach:	They propose a framework that extends Reinforcement Learning with Verifiable Rewards to subjective and open-ended tasks.
Outcome:	The proposed framework improves reasoning capability and response quality on 7 tasks across 50 low-resource languages.

Forge: Quality-Aware Reinforcement Learning for NP-Hard Optimization in LLMs (2026.findings-acl)

Copied to clipboard

Xiaozhe Li, Xinyu Fang, Shengyuan Ding, Yang Li, Linyang Li, Haodong Duan, Qingwen Liu, Kai Chen

Reasoning

Challenge:	Existing benchmarks focus on correctness, overlooking optimality . large language models excel at math, coding, logic and puzzles .
Approach:	They propose a framework for training and evaluating Large Language Models on NP-hard optimization problems through quality-aware RLVR.
Outcome:	The proposed framework outperforms existing benchmarks on math, coding, logic and puzzles.

Don’t Forget Your Reward Values: Language Model Alignment via Value-based Calibration (2024.emnlp-main)

Copied to clipboard

Xin Mao, Feng-Lin Li, Huimin Xu, Wei Zhang, Wang Chen, Anh Tuan Luu

Language Models

Challenge:	Existing methods for generating large language models have been criticized for their complexity and instability.
Approach:	They propose a value-based calibration method to better align Large Language Models with human preferences.
Outcome:	The proposed method surpasses existing methods on AI assistant and summarization datasets, providing impressive generalizability, robustness, and diversity in different settings.

Aligning Large Language Models for Controllable Recommendations (2024.acl-long)

Copied to clipboard

Wensheng Lu, Jianxun Lian, Wei Zhang, Guanghua Li, Mingyang Zhou, Hao Liao, Xing Xie

Language Models

Challenge:	Existing literature focuses on integrating domain-specific knowledge into LLMs to enhance accuracy using a fixed task template.
Approach:	They propose a collection of supervised learning tasks augmented with labels derived from a conventional recommender model to improve LLMs’ proficiency in adhering to recommendation-specific instructions.
Outcome:	The proposed approach significantly improves the capability of LLMs to respond to instructions within recommender systems, reducing formatting errors while maintaining a high level of accuracy.