VIDA: A Visual Intent-driven Design Assistant for Proactive Multimodal Clarification (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing vision-language models fail to provide accurate and complete answers to user requests . a new strategy-aware design assistant is developed to help designers create proactive, visually grounded, and strategically prioritized clarification questions. |
| Approach: | They propose a visual intent-driven design assistant to generate proactive, visually grounded, and strategically prioritized clarification questions. |
| Outcome: | The proposed assistant improves the strategic alignment score by 20.59% over baselines and restores visual grounding capabilities lost during fine-tuning. |
Similar Papers
Scaling Intent Understanding: A Framework for Classification with Clarification using Lightweight LLMs (2026.eacl-industry)
Copied to clipboard
| Challenge: | Proprietary large-language models (LLMs) assign intents to user utterances without addressing ambiguity. |
| Approach: | They propose a domain-agnostic framework that equips open-source LLMs with the ability to perform intent classification and generate clarification questions in case of ambiguity. |
| Outcome: | The proposed framework performs intent classification and generates clarification questions in case of ambiguity. |
AlignMMBench: Evaluating Chinese Multimodal Alignment in Large Vision-Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on basic abilities using nonverbal methods, such as yes-no and multiple-choice questions. |
| Approach: | They propose a benchmark that provides more nuanced evaluations of alignment capabilities for large Vision-Language Models (VLMs) they use a rule-calibrated evaluator that exceeds GPT-4's evaluation ability and a “alignment score” to assess the robustness and stability of models across diverse prompts. |
| Outcome: | The proposed benchmark covers 13 tasks across three categories and includes both single-turn and multi-turn dialogue scenarios. |
MACAROON: Training Vision-Language Models To Be Your Engaged Partners (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large vision-language models (LVLMs) generate detailed responses even when questions are ambiguous or unanswerable, leading to hallucinations and bias issues. |
| Approach: | They propose a three-tiered hierarchy for questions of invalid, ambiguous, and personalizable nature to measure the proactive engagement capabilities of LVLMs. |
| Outcome: | The proposed model generates contrastive response pairs for unlabeled questions, achieving 0.84 AAR, while maintaining comparable performance on general tasks. |
NaturalGAIA: A Verifiable Benchmark and Hierarchical Framework for Long-Horizon GUI Tasks (2026.acl-long)
Copied to clipboard
| Challenge: | Current research faces an "Evaluation-Realism Dilemma" due to unstable MLLM judges or manual verification. |
| Approach: | They propose a verifiable evaluation dataset grounded in real-world human GUI intents. |
| Outcome: | The proposed framework outperforms the state-of-the-art framework in achieving a weighted pathway success rate of 45.6% while reducing token consumption and execution time by 76%. |
ReasonRec: A Reasoning-Augmented Multimodal Agent for Unified Recommendation (2026.findings-acl)
Copied to clipboard
Yihua Zhang, Mingfu Liang, Jiyan Yang, Rong Jin, Wen-Yen Chen, Yiping Han, Huayu Li, Buyun Zhang, Liang Luo, Luke Simon, Sijia Liu, Tianlong Chen, Xi Liu
| Challenge: | Recent advances in multimodal recommenders lack explicit reasoning and self-awareness of uncertainty. |
| Approach: | They propose a reasoning-augmented multimodal agent structured around a three-stage explicit reasoning pipeline. |
| Outcome: | The proposed agent improves ranking metrics and performance on four standard recommendation tasks across five real-world datasets. |
All-in-one: Understanding and Generation in Multimodal Reasoning with the MAIA Benchmark (2025.findings-emnlp)
Copied to clipboard
Davide Testa, Giovanni Bonetta, Raffaella Bernardi, Alessandro Bondielli, Alessandro Lenci, Alessio Miaschi, Lucia Passaro, Bernardo Magnini
| Challenge: | MAIA evaluates visual language models on video-related tasks using reasoning categories that aim to disentangle language and vision relations. |
| Approach: | a native-italian benchmark is designed for fine-grained investigation of the reasoning abilities of visual language models on videos. |
| Outcome: | The benchmark evaluates visual language models on two aligned tasks and a visual question-answering task. |
ProductAgent: Benchmarking Conversational Product Search Agent with Asking Clarification Questions (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Recent advances in conversational information seeking (CIS) suggest a remedy for the lack of interactive clarification when people face unfamiliar domains. |
| Approach: | They propose a fully autonomous conversational information-seeking agent that couples large language models with a set of domain-specific tools to provide product demand clarification. |
| Outcome: | The proposed agent can iterate over 2,000 automatically generated sessions and score high on real-world evaluations without human annotation. |
Visually-Guided Policy Optimization for Multimodal Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing RLVRs lack visual faithfulness due to text-dominated reasoning . a novel framework to reinforce visual focus during policy optimization is proposed . |
| Approach: | They propose a framework to reinforce visual focus during policy optimization using visual attention compensation mechanism. |
| Outcome: | The proposed framework exhibits better visual activation and superior performance in multimodal reasoning and visual-dependent tasks. |
Enhancing Goal-oriented Proactive Dialogue Systems via Dynamic Multi-dimensional Consistency Optimization (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on goal-oriented proactive dialogue systems failed to address the multi-dimensional consistency issue between generated responses and key contextual elements. |
| Approach: | They propose a Dynamic Multi-dimensional Consistency Reinforcement Learning framework which measures the impact of each consistency dimension on overall dialogue quality and provides feedback to improve response quality. |
| Outcome: | The proposed framework significantly improves the consistency of generated responses on two datasets. |
PersonalAlign: Hierarchical Implicit Intent Alignment for Personalized GUI Agent with Long-Term User-Centric Records (2026.acl-long)
Copied to clipboard
| Challenge: | GUI agents have shown strong performance under explicit and completion instructions, but real-world deployment requires aligning with users’ more complex implicit intents. |
| Approach: | They propose a task that requires agents to leverage long-term user records as persistent context to resolve omitted preferences in vague instructions and anticipate latent routines by user state for proactive assistance. |
| Outcome: | The proposed task improves execution and proactive performance by 15.7% and 7.3% under explicit and completion instructions. |