Learning to Generate Structured Output with Schema Reinforcement Learning (2025.acl-long)
Copied to clipboard
Yaxi Lu, Haolun Li, Xin Cong, Zhong Zhang, Yesai Wu, Yankai Lin, Zhiyuan Liu, Fangming Liu, Maosong Sun
| Challenge: | Recent advances in large language models have facilitated the development of intelligent applications like automatic web search (Qin et al., 2023) Several methods exist for generating JSON strings from LLMs, including Prompting but often miss certain schemas. |
| Approach: | They propose to use 40K different JSON schemas to assess models' ability to generate valid JSON outputs. |
| Outcome: | The proposed model improves both in generating JSON outputs and downstream tasks. |
Similar Papers
A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions (2026.acl-long)
Copied to clipboard
Zhiyin Yu, Yuchen Mou, Juncheng Yan, Junyu Luo, Chunchun Chen, Xing Wei, Yunhui Liu, Hongru Sun, Yuxing Zhang, Jun Xu, Yatao Bian, Ming Zhang, Wei Ye, Tieke He, Jie Yang, Guanjie Zheng, Zhonghai Wu, Bo Zhang, Lei Bai, Xiao Luo
| Challenge: | Existing research on reinforcement learning for LLMs under data scarcity has not been unified. |
| Approach: | They propose a top-up hierarchical framework built around three complementary perspectives: data-centric, training-centric and framework-centric. |
| Outcome: | The proposed framework provides a clear conceptual foundation for understanding the design space of data-efficient RL for large language models and to guide researchers working in this emerging area. |
Let Me Speak Freely? A Study On The Impact Of Format Restrictions On Large Language Model Performance. (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Structured generation is used to extract key output information from large language models (LLMs). |
| Approach: | They examine whether constraints on generation space impact LLMs’ abilities, including reasoning and domain knowledge comprehension. |
| Outcome: | The proposed model is based on a few-shot in-context learning and instruction-following capabilities. |
How Good Are LLMs at Processing Tool Outputs? (2026.eacl-long)
Copied to clipboard
Kiran Kate, Yara Rizk, Poulami Ghosh, Ashu Gulati, Tathagata Chakraborti, Zidane Wright, Mayank Agarwal
| Challenge: | Real-world task automation tasks require large language models to call tools, which often return complex JSON responses. |
| Approach: | They evaluated 15 open and closed weight models using multiple prompting approaches to evaluate their tool response processing task and their ability to process structured (JSON) responses. |
| Outcome: | The proposed model can process structured (JSON) responses with 3% to 50% performance differences. |
On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) provide a data-centric solution to alleviate limitations of real-world data with synthetic data generation. |
| Approach: | They propose a generic workflow for LLM-driven synthetic data generation. |
| Outcome: | The proposed workflows highlight gaps in existing research and outline avenues for future studies. |
Evaluating Language Models as Synthetic Data Generators (2025.acl-long)
Copied to clipboard
Seungone Kim, Juyoung Suk, Xiang Yue, Vijay Viswanathan, Seongyun Lee, Yizhong Wang, Kiril Gashteovski, Carolin Lawrence, Sean Welleck, Graham Neubig
| Challenge: | Prior studies have focused on developing effective data generation methods, but lack systematic comparison of different LMs as data generators in a unified setting. |
| Approach: | They propose to use a benchmark to compare language models' data generation abilities against a set of standardized settings and metrics. |
| Outcome: | The proposed benchmark provides standardized settings and metrics to evaluate LMs’ data generation abilities. |
Reference-Free Schema Generation for Literature Review Tables via Multi-Faceted Rewards (2026.acl-srw)
Copied to clipboard
| Challenge: | Literature review systems generate literature review tables by inferring schemas and values from documents. |
| Approach: | They propose to use schema generation as a reinforcement learning problem to determine which dimensions to compare a set of papers. |
| Outcome: | The proposed model improves over the untuned model across intrinsic, reference-based, and LLM-judge metrics and remains competitive with supervised fine-tune models at 5 the parameter count on structural and diversity dimensions. |
Adaptive Constraint Propagation: Scaling Structured Inference for Large Language Models via Meta-Reinforcement Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models increasingly require structured inference, says aaron sagar . meta-learning learns universal constraint propagation policies without task-specific training . standard schedulers are inexpensive but myopic, because they optimize local effects . |
| Approach: | MetaJuLS learns universal constraint propagation policies applicable across languages and tasks without task-specific retraining. |
| Outcome: | MetaJuLS achieves 1.5-2.0 speedups over GPU-optimized baselines while maintaining accuracy within 0.2% of state-of-the-art parsers. |
DocCGen: Document-based Controlled Code Generation (2024.emnlp-main)
Copied to clipboard
Sameer Pimparkhede, Mehant Kammakomati, Srikanth Tamilselvam, Prince Kumar, Ashok Kumar, Pushpak Bhattacharyya
| Challenge: | Large language models (LLMs) produce state-of-the-art performance on natural language to code generation for resource-rich general-purpose languages like C++, Java, and Python. |
| Approach: | They propose a framework that breaks the NL-to-Code generation task into two steps . they use library documentation to detect the correct libraries and schema rules extracted from the documentation to constrain the decoding . |
| Outcome: | The proposed framework improves different sized language models across all six evaluation metrics, reducing syntactic and semantic errors in structured code. |
Adaptive Reinforcement Tuning Language Models as Hard Data Generators for Sentence Representation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods use contrastive learning (CL) to learn effective sentence representations, but require extensive human annotation. |
| Approach: | They propose a reinforcement learning approach for fine-tuning small-parameter LLMs to generate high-quality hard contrastive data without human feedback. |
| Outcome: | The proposed method achieves state-of-the-art on seven semantic text similarity tasks. |
StepCoder: Improving Code Generation with Reinforcement Learning from Compiler Feedback (2024.acl-long)
Copied to clipboard
Shihan Dou, Yan Liu, Haoxiang Jia, Enyu Zhou, Limao Xiong, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, Yuhao Zhou, Tao Ji, Rui Zheng, Qi Zhang, Tao Gui, Xuanjing Huang
| Challenge: | Existing work integrates reinforcement learning with compiler feedback to enhance code generation quality but the long code generated by LLMs makes RL exploration ineffective. |
| Approach: | They propose a framework that integrates reinforcement learning and compiler feedback to enhance code generation quality. |
| Outcome: | The proposed framework outperforms state-of-the-art approaches in corresponding benchmarks and integrates reinforcement learning with compiler feedback to improve code generation quality. |