Mete Ismayilzada, Antonio Laverghetta Jr., Simone A. Luchini, Reet Patel, Antoine Bosselut, Lonneke Van Der Plas, Roger E. Beaty
| Challenge: | Existing methods for enhancing LLM creativity focus on diversity or specific tasks, failing to address creativity’s multifaceted nature in a generalizable way. |
| Approach: | They propose a method that injects signals from multiple creativity dimensions into the preference optimization objective in a modular fashion. |
| Outcome: | The proposed method outperforms baseline models on automated and human evaluations while maintaining high output quality. |
Similar Papers
Bridging the Creativity Understanding Gap: Small-Scale Human Alignment Enables Expert-Level Humor Ranking in LLMs (2025.findings-emnlp)
Copied to clipboard
Kuan Lok Zhou, Jiayi Chen, Siddharth Suresh, Reuben Narad, Timothy T. Rogers, Lalit K Jain, Robert D Nowak, Bob Mankoff, Jifan Zhang
| Challenge: | Large Language Models (LLMs) have shown significant limitations in understanding creative content, as demonstrated by Hessel et al. (2023)’s influential work on the New Yorker Cartoon Caption Contest. |
| Approach: | They propose to decompose humor understanding into three components and improve each by enhancing visual understanding through improved annotation and utilizing LLM-generated humor reasoning and explanations. |
| Outcome: | The proposed approach achieves 82.4% accuracy in caption ranking, significantly better than the previous 67% benchmark and matches the performance of world-renowned human experts in this domain. |
Aligning LLMs with Individual Preferences via Interaction (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies on LLMs alignment focus on generalizing their behavior to generalized values such as helpfulness, harmlessness, and honesty. |
| Approach: | They train large language models to "interact to align" to implicitly infer user preferences . they use a multi-turn preference dataset to generate a personalized alignment . |
| Outcome: | The proposed method enables dynamic, personalized alignment via interaction with a multi-turn preference dataset. |
🧑🍳 Cooking Up Creativity: Enhancing LLM Creativity through Structured Recombination (2026.tacl-1)
Copied to clipboard
| Challenge: | Large Language Models excel at many tasks, yet struggle to generate truly creative ideas. |
| Approach: | They propose a novel approach that enhances Large Language Models' creativity by manipulating structured representations of existing ideas. |
| Outcome: | The proposed model outperforms GPT-4o in novelty and diversity and outperformed GPT-0 in creative generation. |
Disentangling Preference Representation and Text Generation for Efficient Individual Preference Alignment (2025.coling-main)
Copied to clipboard
| Challenge: | Human values are inherently diverse, making it insufficient to align LLMs solely with general preferences. |
| Approach: | They propose a flexible paradigm for individual preference alignment that disentangles preference representation from text generation in LLMs. |
| Outcome: | The proposed method produces aligned quality and better than PEFT-based methods while reducing training time for each new individual preference by 80% to 90%. |
Insights into Alignment: Evaluating DPO and its Variants Across Multiple Tasks (2025.acl-srw)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel in math reasoning problemsolving, text generation, summarization, creative writing, among other tasks. |
| Approach: | They evaluate Direct Preference Optimization and its variants for aligning Large Language Models with human preferences. |
| Outcome: | The proposed alignment methods achieve near-optimal performance even with smaller subsets of training data. |
CRPO: Confidence-Reward Driven Preference Optimization for Machine Translation (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown great potential in natural language processing tasks, but their application to machine translation remains challenging due to pretraining on predominantly English-centric datasets. |
| Approach: | They propose a method that combines reward scores with model confidence to improve model selection for fine-tuning. |
| Outcome: | The proposed method outperforms existing methods in translation accuracy and data efficiency. |
From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignment (2026.acl-long)
Copied to clipboard
| Challenge: | Current approaches to align large language models assume uniform human preferences, overlooking the diversity inherent in human populations. |
| Approach: | They propose a framework for scalable personalized alignment of large language models . they establish a preference space characterizing psychological and behavioral dimensions . |
| Outcome: | The proposed framework improves on existing methods with an average of 17.06% accuracy gain across four benchmarks and a strong adaptation capability to novel preferences. |
Aligning Large Language Models with Implicit Preferences from User-Generated Content (2025.acl-long)
Copied to clipboard
Zhaoxuan Tan, Zheng Li, Tianyi Liu, Haodong Wang, Hyokun Yun, Ming Zeng, Pei Chen, Zhihan Zhang, Yifan Gao, Ruijie Wang, Priyanka Nigam, Bing Yin, Meng Jiang
| Challenge: | Existing preference learning methods rely heavily on curated data from humans or advanced LLMs, which is costly and difficult to scale. |
| Approach: | They propose a framework that leverages implicit preferences in unlabeled user-generated content to generate preference data. |
| Outcome: | The proposed framework transforms user-generated content into user queries and generates responses from the policy model. |
SeaPO: Strategic Error Amplification for Robust Preference Optimization of Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for preference optimization of large language models use pairs of positive and negative samples, but the quality of positive samples may become similar during training, complicating preference learning. |
| Approach: | SeaPO introduces error types commonly occurring in large language models to improve preference learning. |
| Outcome: | SeaPO introduces error types into model Preference Optimization to improve model performance . negative samples are more erroneous than positive samples, and preference-based training mitigates errors . |
Dissecting Human and LLM Preferences (2024.acl-long)
Copied to clipboard
| Challenge: | a recent study shows that human and Large Language Model preferences are important for model fine-tuning and evaluation. |
| Approach: | They dissect the preferences of human and 32 different Large Language Models to understand their quantitative composition. |
| Outcome: | The proposed model is compared with 32 different large language models using real-world user-model conversations. |