EvolvR: Self-Evolving Pairwise Reasoning for Story Evaluation to Enhance Generation (2026.acl-long)
Copied to clipboard
Xinda Wang, Zhengxu Hou, Yangshijie Zhang, null Yanbingren, Jialin Liu, ChenZhuo Zhao, Zhibo Yang, Bin-Bin Yang, Feng Xiao
| Challenge: | Existing methods for story evaluation lack reasoning capabilities for open-source models . evolvR framework provides high-fidelity evaluators for story generation tasks . |
| Approach: | They propose a framework that self-synthesizes chain-of-thought data via a multi-persona strategy . they propose evolvR to provide a reward model for story generation . |
| Outcome: | The proposed framework achieves state-of-the-art performance on three evaluation benchmarks . it also enhances the quality of generated stories, validating the superiority of the framework . |
Similar Papers
EvoCoT: Overcoming the Exploration Bottleneck in Reinforcement Learning for LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to reinforcement learning with verifiable reward (RLVR) are limited by difficulty or lack of exploration. |
| Approach: | They propose a self-evolving curriculum learning framework based on chain-of-thought reasoning optimization that constrains exploration space by self-generating and verifying CoT trajectories. |
| Outcome: | The proposed framework enables LLMs to solve previously unsolved problems without external supervision and is compatible with various RL fine-tuning methods. |
Choose Your Own Adventure: Paired Suggestions in Collaborative Writing for Evaluating Story Generation Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Comparative evaluations have been shown to produce more reliable and consistent results than Likert scale ratings. |
| Approach: | They propose a collaborative writing setup where two models generate suggestions to people as they write a short story and then ask them to choose which model's suggestions they prefer. |
| Outcome: | The proposed model performs better in cases where the differences in generation methods are small (nucleus vs. top-k sampling) and large (GPT2 v. Fusion models). |
StoryER: Automatic Story Evaluation via Ranking, Rating and Reasoning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing automatic story evaluation methods place a premium on story lexical level coherence, deviating from human preference. |
| Approach: | They propose a novel Story Evaluation method that mimics human preference when judging a story . the model is based on a well-annotated dataset and a longformer-encoder-decoder . |
| Outcome: | The proposed method is applicable to machine-generated and human-written stories. |
Text-to-Text Automatic Story Generation: A Survey (2026.eacl-srw)
Copied to clipboard
| Challenge: | Automated story generation aims to produce coherent, engaging, and contextually consistent narratives with minimal or no human involvement . despite advances in large language models, maintaining narrative coherence, character consistency, storyline diversity, and plot controllability in generating stories is still challenging. |
| Approach: | They propose to develop new evaluation metrics and better data sets to support automatic story generation. |
| Outcome: | The proposed evaluation metrics and better datasets will improve narrative coherence and consistency and explore practical applications of story generation. |
Plot-guided Adversarial Example Construction for Evaluating Open-domain Story Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to generate implausible stories using plots are unnatural and oversimplify the characteristics of implusible machine-generated stories. |
| Approach: | They propose to generate a more comprehensive set of implausible stories using plots . plots are structured representations of controllable factors used to generate stories . |
| Outcome: | The proposed model improves the quality of generated implausible stories using plots . it shows that the evaluation metrics trained on the generated data correlate better with human judgments compared to baselines. |
EvoSci: A Bio-Inspired Multi-Agent Framework for the Evolution of Scientific Discovery (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for large language models (LLMs) face challenges in the design of research workflows and multi-role collaboration mechanisms. |
| Approach: | They propose a multi-agent scientific collaboration framework which integrates bio-inspired evolution with knowledge graph modeling. |
| Outcome: | EvoSci outperforms baselines in peer-review and ranking evaluations on real-world research topics. |
STORYTELLER: An Enhanced Plot-Planning Framework for Coherent and Cohesive Story Generation (2025.findings-acl)
Copied to clipboard
Jiaming Li, Yukun Chen, Ziqiang Liu, Minghuan Tan, Lei Zhang, Yunshui Li, Run Luo, Longze Chen, Jing Luo, Ahmadreza Argha, Hamid Alinejad-Rokny, Wei Zhou, Min Yang
| Challenge: | Existing methods for storytelling lack coherence and consistency, compromising the overall storytelling experience. |
| Approach: | They propose a novel approach that improves the coherence and consistency of automatically generated stories by managing plot nodes and enabling dynamic interactions between different parts of the story. |
| Outcome: | The proposed approach outperforms existing methods in 84.33% of the trials. |
STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets lack rich enough contexts to guide models and evaluations are unreliable for long-form creative text. |
| Approach: | They propose a dataset and evaluation platform built from STORIUM . their dataset contains 6K lengthy stories with fine-grained natural language annotations . |
| Outcome: | The proposed model can be used to generate 6K long stories with fine-grained natural language annotations and a user-generated dataset. |
Auto-Evolve: Enhancing Large Language Model’s Performance via Self-Reasoning Framework (2024.findings-emnlp)
Copied to clipboard
Krishna Aswani, Huilin Lu, Pranav Patankar, Priya Dhalwani, Xue Tan, Jayant Ganeshmohan, Simon Lacasse
| Challenge: | Recent advances in prompt engineering strategies rely on static seed reasoning modules to simulate human approach to problem-solving. |
| Approach: | They propose a framework that enables LLMs to self-create dynamic reasoning modules and downstream action plan. |
| Outcome: | The proposed framework outperforms existing prompting strategies on a BigBench-Hard dataset and improves performance by 2.8% over existing methods. |
Praetor: A Fine-Grained Generative LLM Evaluator with Instance-Level Customizable Evaluation Criteria (2025.acl-long)
Copied to clipboard
Yongqi Leng, Renren Jin, Yue Chen, Zhuowen Han, Ling Shi, Jianxiang Peng, Lei Yang, Juesi Xiao, Deyi Xiong
| Challenge: | Existing evaluation methods are inadequate to evaluate large language models (LLMs). |
| Approach: | They propose a fine-grained generative LLM evaluator with instance-level customazable evaluation criteria that can be used to evaluate large language models. |
| Outcome: | The proposed model outperforms existing LLM evaluators and instruction-tuned LLMs on multiple benchmarks and sets new SOTA results. |