GRAVITY: A Framework for Personalized Text Generation via Profile-Grounded Synthetic Preferences

Challenge:	Personalization in LLMs often relies on costly human feedback or interaction logs, limiting scalability and neglecting deeper user attributes.
Approach:	They propose a framework for generating synthetic, profile-grounded preference data that captures users’ interests, values, beliefs, and personality traits.
Outcome:	The proposed framework improves on book descriptions for 400 Amazon users across multiple cultures, with user studies showing that outputs are preferred over 86% of the time.

Similar Papers

Overview

Challenge

Approach

Outcome

A Grounded Preference Model for LLM Alignment (2024.findings-acl)

Copied to clipboard

Tahira Naseem, Guangxuan Xu, Sarathkrishna Swaminathan, Asaf Yehudai, Subhajit Chaudhury, Radu Florian, Ramón Astudillo, Asim Munawar

Language Models

Challenge:	Large Language Models (LLMs) suffer from factual inconsistency and hallucination despite recent advances . training a preference model requires substantial human annotation, which is expensive and labor-intensive.
Approach:	They propose to generate synthetic grounded preference data and train a Grounded Preference Model to assess the overall quality of grounded responses.
Outcome:	The proposed model can generate much better grounded responses as judged by GPT4 and achieves the TRUE faithfulness Benchmark.

Synthia: Scalable Grounded Persona Generation from Social Media Data (2026.acl-long)

Copied to clipboard

Vahid Rahimzadeh, Erfan Moosavi Monazzah, Mohammad Taher Pilehvar, Yadollah Yaghoobzadeh

Challenge:	Persona-driven large language models (LLMs) are increasingly used in computational social science, yet their validity critically depends on the fidelity of the underlying personas.
Approach:	They propose a persona-generation framework that grounds LLM-generated personas in real social-media posts while delegating narrative construction to language models.
Outcome:	The proposed framework outperforms state-of-the-art methods for most demographics across different dimensions while maintaining interaction graph structure among personas grounded in real social network users.

From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignment (2026.acl-long)

Copied to clipboard

Jia-Nan Li, Jian Guan, Songhao Wu, Wei Wu, Rui Yan

Robustness in NLP

Challenge:	Current approaches to align large language models assume uniform human preferences, overlooking the diversity inherent in human populations.
Approach:	They propose a framework for scalable personalized alignment of large language models . they establish a preference space characterizing psychological and behavioral dimensions .
Outcome:	The proposed framework improves on existing methods with an average of 17.06% accuracy gain across four benchmarks and a strong adaptation capability to novel preferences.

SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMs (2025.acl-long)

Copied to clipboard

Michael J Ryan, Omar Shaikh, Aditri Bhagirath, Daniel Frees, William Barr Held, Diyi Yang

Low-Resource NLP

Challenge:	Recent calls for pluralistic alignment of Large Language Models encourage adapting models to diverse user preferences.
Approach:	They propose a method to induce synthetic user personas from user interactions for personalized reward modeling.
Outcome:	The proposed approach improves LLM-as-a-judge accuracy by 4.4% on Chatbot Arena.

Aligning Large Language Models with Implicit Preferences from User-Generated Content (2025.acl-long)

Copied to clipboard

Zhaoxuan Tan, Zheng Li, Tianyi Liu, Haodong Wang, Hyokun Yun, Ming Zeng, Pei Chen, Zhihan Zhang, Yifan Gao, Ruijie Wang, Priyanka Nigam, Bing Yin, Meng Jiang

Language Models

Challenge:	Existing preference learning methods rely heavily on curated data from humans or advanced LLMs, which is costly and difficult to scale.
Approach:	They propose a framework that leverages implicit preferences in unlabeled user-generated content to generate preference data.
Outcome:	The proposed framework transforms user-generated content into user queries and generates responses from the policy model.

WildFeedback: Aligning LLMs With In-situ User Interactions And Feedback (2026.acl-long)

Copied to clipboard

Taiwei Shi, Zhuoer Wang, Longqi Yang, Ying-Chun Lin, Zexue He, Mengting Wan, Pei Zhou, Sujay Kumar Jauhar, Sihao Chen, Shan Xia, Hongfei Zhang, Jieyu Zhao, Xiaofeng Xu, Xia Song, Jennifer Neville

Challenge:	Traditional alignment methods rely on human annotations and are subjective and misalignment with real-world user preferences.
Approach:	They propose a framework that leverages in-situ user feedback during conversations with LLMs to create preference datasets automatically.
Outcome:	The proposed framework identifies and classifies user feedback to LLM responses between conversation turns and creates examples of preferred and dispreferred responses according to user preferences.

Guided Profile Generation Improves Personalization with Large Language Models (2024.findings-emnlp)

Copied to clipboard

Jiarui Zhang

Language Models

Challenge:	Existing approaches to personalization with LLMs rely on sparse and complex personal contexts, resulting in incomplete interpretation.
Approach:	They propose a general method to generate personal profiles in natural language that extracts important, distinctive features from the personal context into concise, descriptive sentences.
Outcome:	The proposed method improves personalization ability across different tasks, for example, it increases 37% accuracy in predicting personal preference compared to directly feeding the LLMs with raw personal context.

BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Data (2025.acl-long)

Copied to clipboard

Wenkai Li, Jiarui Liu, Andy Liu, Xuhui Zhou, Mona T. Diab, Maarten Sap

Language Models

Challenge:	Existing methods for embedding human personality traits into LLMs are limited by realism and validity issues.
Approach:	They propose to use a large-scale dataset to embed human personality traits into LLMs . they use supervised fine-tuning and direct preference optimization to train LLM models .
Outcome:	The proposed methods outperform prompting on personality assessments and IPIP-NEO, and show higher conscientiousness, agreeableness, lower extraversion, and lower neuroticism on reasoning tasks.

On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey (2024.findings-acl)

Copied to clipboard

Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, Haobo Wang

Challenge:	Large Language Models (LLMs) provide a data-centric solution to alleviate limitations of real-world data with synthetic data generation.
Approach:	They propose a generic workflow for LLM-driven synthetic data generation.
Outcome:	The proposed workflows highlight gaps in existing research and outline avenues for future studies.

When Personalization Meets Reality: A Multi-Faceted Analysis of Personalized Preference Learning (2025.findings-emnlp)

Copied to clipboard

Yijiang River Dong, Tiancheng Hu, Yinhong Liu, Ahmet Üstün, Nigel Collier

Challenge:	Reinforcement Learning from Human Feedback assumes homogeneous preferences across users . personalization can introduce up to 20% safety misalignment .
Approach:	They propose a framework to assess personalized preference learning by tailoring preferences for users . they compare eight personalization methods across three preference datasets .
Outcome:	The proposed framework measures performance, fairness, unintended effects, adaptability across preferences . performance differences between personalization methods could reach 36% when users strongly disagree .