Natural-Language Policies to Executable Decisions: An Interpretable Large Language Model Framework (2026.acl-industry)
Copied to clipboard
Ziqiang Zhang, Jing Ma, Zilong Wang, Jiayuan Chen, Yi Qiao, Yu He, Wei Zhang, Dai Cheng, Xiaoyu Shen
| Challenge: | a production-grade pricing system for tourism is challenging due to unstructured nature of travel orders and ever-evolving pricing policies. |
| Approach: | They propose a production-grade pricing system with a strict decision boundary . they propose to combine structured extraction and bounded policy/path selection with interpretable condition trees . |
| Outcome: | The proposed system processed 3,960 orders in six months and reduced the order management team from 15-20 to 3 . the system reduced the per-order handling time from 10 minutes to 2 minutes. |
Similar Papers
PricingLogic: Evaluating LLMs Reasoning on Complex Tourism Pricing Tasks (2025.emnlp-main)
Copied to clipboard
Yunuo Liu, Dawei Zhu, Zena Al-Khalili, Dai Cheng, Yanjun Chen, Dietrich Klakow, Wei Zhang, Xiaoyu Shen
| Challenge: | Recent advances in Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse domains, such as code generation, mathematical problem-solving, and general-purpose human instruction following. |
| Approach: | They propose to use large language models to process questions expressed in natural language to automate tourism-booking prices when multiple, overlapping farerules apply. |
| Outcome: | The proposed model can automate tourism-booking prices when multiple, overlapping farerules apply. |
RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarios (2025.acl-long)
Copied to clipboard
| Challenge: | RuleArena assesses the ability of large language models (LLMs) to follow complex, real-world rules in reasoning. |
| Approach: | They propose a benchmark to evaluate the ability of large language models (LLMs) to follow complex, real-world rules in reasoning. |
| Outcome: | The proposed benchmark covers airline baggage fees, NBA transactions, and tax regulations. |
ModelingAgent: Bridging LLMs and Mathematical Modeling for Real-World Challenges (2025.findings-emnlp)
Copied to clipboard
Cheng Qian, Hongyi Du, Hongru Wang, Xiusi Chen, Yuji Zhang, Avirup Sil, ChengXiang Zhai, Kathleen McKeown, Heng Ji
| Challenge: | Existing benchmarks for large language models fail to reflect real-world complexity . existing benchmarks often fail to capture real-life problems . |
| Approach: | They propose a benchmark that features real-world-inspired, open-ended problems from competitions . they propose 'ModelingBench' that supports multiple valid solutions . |
| Outcome: | The proposed framework outperforms baselines and produces well-grounded, creative solutions. |
LLMs as Planning Formalizers: A Survey for Leveraging Large Language Models to Construct Automated Planning Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models excel in various natural language tasks but struggle with long-horizon planning problems requiring structured reasoning. |
| Approach: | They propose to integrate large language models into AP and NLP planning frameworks by reviewing current research and identifying critical challenges and future directions. |
| Outcome: | The proposed frameworks are used to support reliable off-the-shelf AP planners. |
GRAMMAR-LLM: Grammar-Constrained Natural Language Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to fine-tuning and prompting are insufficient to ensure compliance with predefined taxonomies, syntactic structures, or domain-specific rules. |
| Approach: | They propose a framework that integrates formal grammatical constraints into the decoding process to enforce syntactic correctness in linear time while maintaining expressiveness in grammar rule definition. |
| Outcome: | The proposed framework enforces syntactic correctness in linear time while maintaining expressiveness in grammar rule definition. |
Noise, Adaptation, and Strategy: Assessing LLM Fidelity in Decision-Making (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used for social science simulations . however, most evaluations focus on task optimality rather than variability and adaptation characteristic of human decision-making. |
| Approach: | They propose a process-oriented evaluation framework with progressive interventions to evaluate two economics tasks using large language models. |
| Outcome: | The proposed evaluation framework targets two economic tasks with progressive interventions. |
StitchLLM: Serving LLMs, One Block at a Time (2025.acl-long)
Copied to clipboard
Bodun Hu, Shuozhe Li, Saurabh Agarwal, Myungjin Lee, Akshay Jajoo, Jiamin Li, Le Xu, Geon-Woo Kim, Donghyun Kim, Hong Xu, Amy Zhang, Aditya Akella
| Challenge: | Existing techniques like distillation and pruning are not efficient for large language models. |
| Approach: | They propose a dynamic model routing framework that uses a powerful bottom model to process all queries and a lightweight routing mechanism to allocate computational resources appropriately. |
| Outcome: | The proposed framework improves system throughput while minimizing performance degradation. |
Language Model as Planner and Formalizer under Constraints (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been widely used in planning but lack interpretability and control. |
| Approach: | They propose to augment widely used planning benchmarks with manually annotated, fine-grained, and rich natural language constraints spanning four formally defined categories. |
| Outcome: | The proposed model outperforms existing models in 4 state-of-the-art reasoning LLMs, 4 formal languages, and 4 datasets. |
Calibrating LLM-Based Evaluator (2024.lrec-main)
Copied to clipboard
Yuxuan Liu, Tianchi Yang, Shaohan Huang, Zihan Zhang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, Qi Zhang
| Challenge: | Existing models for large language models lack the ability to calibrate their outputs towards human preference. |
| Approach: | They propose a multi-stage, gradient-free approach to calibrate an LLM-based evaluator toward human preference. |
| Outcome: | The proposed approach improves correlation with expert evaluation on multiple text quality evaluation datasets. |
Business as Rulesual: A Benchmark and Framework for Business Rule Flow Modeling with LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for extracting structured procedural knowledge from unstructured business documents are limited by simplistic schemas and shallow logical dependencies. |
| Approach: | They propose a framework for extracting structured procedural knowledge from unstructured business documents . they propose BREX, a carefully curated benchmark comprising 409 real-world business documents and 2,855 expert-annotated rules . |
| Outcome: | The proposed framework outperforms standard prompts in rule extraction and execution. |