AlignCap: Aligning Speech Emotion Captioning to Human Preferences (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for speech emotion capture often produce hallucinations and lose generalization on unseen speech. |
| Approach: | They propose to align speech emotion captioning to human preference based on large language model (LLM) and human preference regularization to eliminate factuality and faithfulness hallucinations. |
| Outcome: | Experiments show that AlignCap performs better than existing methods on Zero-shot SEC task. |
Similar Papers
Aligning Language Models to User Opinions (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Personality is a defining feature of human beings, shaped by a complex interplay of demographic characteristics, moral principles, and social experiences. |
| Approach: | They use public opinion surveys to model past user opinions in addition to user demographics and ideology to achieve up to 7 points accuracy gains in predicting public opinions from survey questions. |
| Outcome: | The proposed model achieves 7 points accuracy gains in predicting public opinions from public opinion surveys across a broad set of topics. |
Aligning Black-box Language Models with Human Judgments (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used as automated judges to evaluate recommendation systems, search engines, and other subjective tasks. |
| Approach: | They propose a framework to align LLM judgments with individual human evaluators or their aggregated judgments without retraining or fine-tuning the LLM. |
| Outcome: | The proposed framework achieves 142% improvement in agreement across 29 tasks and exceeds inter-human agreement on four out of six tasks. |
Beyond Silent Letters: Amplifying LLMs in Emotion Recognition with Vocal Nuances (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that Large Language Models possess a form of emotional intelligence, capable of interpreting emotional stimuli in text. |
| Approach: | They propose a method that translates speech characteristics into natural language descriptions and integrates them into LLMs to perform multimodal emotion analysis via text prompts. |
| Outcome: | The proposed method outperforms baseline models that require structural modifications on two datasets showing significant improvements in emotion recognition accuracy. |
From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignment (2026.acl-long)
Copied to clipboard
| Challenge: | Current approaches to align large language models assume uniform human preferences, overlooking the diversity inherent in human populations. |
| Approach: | They propose a framework for scalable personalized alignment of large language models . they establish a preference space characterizing psychological and behavioral dimensions . |
| Outcome: | The proposed framework improves on existing methods with an average of 17.06% accuracy gain across four benchmarks and a strong adaptation capability to novel preferences. |
Comparing Bad Apples to Good Oranges Aligning Large Language Models via Joint Preference Optimization (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that acquiring human preferences by comparing generations is not effective for large language models. |
| Approach: | They propose a preference optimization objective that elicits preferences jointly over the instruction-response pairs. |
| Outcome: | The proposed approach outperforms prior preference optimizations by 5.2% and 3.3% in summarization and open-ended dialogue datasets. |
Can Large Language Models Capture Dissenting Human Voices? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown impressive achievements in solving a broad range of tasks. |
| Approach: | They evaluate the performance and alignment of large language models with humans using Monte Carlo Estimation and Log Probability Estimationic methods to estimate the multinomial distribution. |
| Outcome: | The proposed models fail to capture human disagreement distribution and inference and human alignment performance plunge even further on data samples with high disagreement levels raising concerns about their natural language understanding ability and representativeness to a larger human population. |
Improving Alignment in LVLMs with Debiased Self-Judgment (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for aligning LVLMs rely on external datasets, human annotations or complex post-processing. |
| Approach: | They propose a method that generates a debiased self-judgment score for LVLMs . this self-evaluation metric is created internally by the model without external resources . |
| Outcome: | The proposed approach outperforms existing methods in reducing hallucinations and safety concerns. |
Generative Error Correction for Emotion-aware Speech-to-text Translation (2025.findings-acl)
Copied to clipboard
| Challenge: | Despite recent advances in speech-to-text translation, the impact of the emotion content has been overlooked. |
| Approach: | They propose to use generative error correction (GER) to generate the translation based on the decoded N-best hypotheses and combine emotion and sentiment labels into the LLM finetuning process to enable the model to consider the emotion content. |
| Outcome: | The proposed model can translate speech in English-Chinese using GER and emotion and sentiment labels. |
Alignment Data Map for Efficient Preference Data Selection and Diagnosis (2026.findings-acl)
Copied to clipboard
| Challenge: | constructing high-quality preference datasets faces scalability challenges due to prohibitive cost and complexity of human annotation. |
| Approach: | They propose a tool to identify and select effective preference data by LLM-as-a-judge, explicit reward model, and reference-based approaches. |
| Outcome: | The proposed tool reduces annotation costs while preserving alignment effectiveness. |
Would you Rather? A New Benchmark for Learning Machine Alignment with Cultural Values and Social Preferences (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies on optimal decision-making are limited and only consider individuals in isolation. |
| Approach: | They propose a task and corpus for learning alignments between machine and human preferences based on a gamified voting game . |
| Outcome: | The proposed task and corpus show that current state-of-the-art NLP models still leave much room for improvement. |