Papers by Yuyu Luo
Flattery in Motion: Benchmarking and Analyzing Sycophancy in Video-LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Current sycophancy research has largely overlooked its specific manifestations in the video-language domain. |
| Approach: | They propose a video-LLM sycophancy benchmarking and evaluation to evaluate scophancies in video-LLMs. |
| Outcome: | The proposed benchmark evaluates sycophantic behavior in state-of-the-art Video-LLMs across diverse question formats, prompt biases, and visual reasoning tasks. |
Data Interpreter: An LLM Agent for Data Science (2025.findings-acl)
Copied to clipboard
Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Li Zhang, Lingyao Zhang, Min Yang, Mingchen Zhuge, Taicheng Guo, Tuo Zhou, Wei Tao, Robert Tang, Xiangtao Lu, Xiawu Zheng, Xinbing Liang, Yaying Fei, Yuheng Cheng, Yongxin Ni, Zhibin Gou, Zongze Xu, Yuyu Luo, Chenglin Wu
| Challenge: | Large Language Models (LLMs) excel in various domains but face challenges when applied to data science workflows due to their complex, multi-stage nature. |
| Approach: | They propose a hierarchical graph-based agent that represents complexity and a progressive strategy for step-by-step verification, refinement, and consistent context management. |
| Outcome: | The proposed agent surpasses state-of-the-art baselines on the MATH dataset and performs better on InfiAgent-DABench. |
DPC: Training-Free Text-to-SQL Candidate Selection via Dual-Paradigm Consistency (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generating SQL queries lack the ability to self-evaluate correctness without an execution oracle. |
| Approach: | They propose a framework that reformulates SQL selection from a probabilistic guessing task on hidden data into a deterministic verification task on visible data. |
| Outcome: | Experiments on BIRD and Spider show that the proposed method outperforms baselines. |
Self-Supervised Prompt Optimization (2025.findings-emnlp)
Copied to clipboard
Jinyu Xiang, Jiayi Zhang, Zhaoyang Yu, Xinbing Liang, Fengwei Teng, Jinhao Tu, Fashen Ren, Xiangru Tang, Sirui Hong, Chenglin Wu, Yuyu Luo
| Challenge: | Existing prompt optimization methods rely heavily on external references such as ground truth or by humans, limiting their applicability in real-world scenarios where such data is unavailable or costly to obtain. |
| Approach: | They propose a cost-efficient framework that discovers effective prompts for both closed and open-ended tasks without external reference. |
| Outcome: | The proposed framework outperforms state-of-the-art prompt optimization methods with significantly lower costs and fewer samples. |
IGenBench: Benchmarking the Reliability of Text-to-Infographic Generation (2026.acl-long)
Copied to clipboard
Yinghao Tang, Xueding Liu, Boyuan Zhang, Tingfeng Lan, Yupeng Xie, Jiale Lao, Yiyao Wang, Haoxuan Li, Tingting Gao, Bo Pan, Luoxuan Weng, Xiuqi Huang, Minfeng Zhu, Yingchaojie Feng, Yuyu Luo, Wei Chen
| Challenge: | Generated infographics may appear correct at first glance but contain easily overlooked issues, such as distorted data encoding or incorrect textual content. |
| Approach: | They propose to evaluate reliability of text-to-infographic generation using IGenBench . they employ multimodal large language models to verify each question . |
| Outcome: | The proposed framework decomposes reliability verification into atomic yes/no questions based on a taxonomy of 10 question types. |
MEBench: Benchmarking Large Language Models for Cross-Document Multi-Entity Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) and Retrieval-augmented Generation (RAG) systems show promise, but their performance on cross-document MEQA remains underexplored due to the lack of tailored benchmarks. |
| Approach: | They propose a scalable multi-document, multi-entity benchmark to evaluate LLMs' capacity to retrieve, consolidate, and reason over scattered and dense information. |
| Outcome: | The proposed benchmarks show that even advanced models achieve only 59% accuracy on MEBench. |
ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answering (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Chart question answering (ChartQA) tasks are a critical part of visualization charts. |
| Approach: | They propose a chart question answering task that uses MLLMs to analyze charts . they propose 'Chain-of-Charts' textual prompt strategy that directs attention to visual elements . |
| Outcome: | The proposed model improves performance by 14.41% and 80% in low-level ChartQA tasks. |
MAR: Matching-Augmented Reasoning for Enhancing Visual-based Entity Question Answering (2024.emnlp-main)
Copied to clipboard
| Challenge: | Multimodal large language models (MLLMs) struggle with visual-based entity questions (VEQA) MLLM can identify A, but may refrain from answering due to privacy concerns. |
| Approach: | They propose a method that uses vector representations to analyze visual-based entity questions (VEQA) they use visual cues and textual information to integrate visual cus and visual information . |
| Outcome: | The proposed method significantly improves visual-based entity question answering (VEQA) it can identify faces, names, and alignments within visual objects, and then derive the answer over this matching graph. |
ROSE: An Intent-Centered Evaluation Metric for NL2SQL (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation metrics for evaluating the effectiveness of Natural Language to SQL (NL2SQL) solutions are becoming unreliable due to its sensitiveness to syntactic variation and inconsistent consistency with ground-truth SQL. |
| Approach: | They propose an intent-centered metric that focuses on whether the predicted SQL answers the question, rather than consistency with the ground-truth SQL. |
| Outcome: | The proposed metric outperforms the next-best metric by nearly 24% on the expert-aligned validation set **ROSE-VEC**. |
Concise Math Reasoning via Difficulty-Aware Distillation (2026.findings-acl)
Copied to clipboard
Yifan Wu, Jingze Shi, Bingheng Wu, Jiayi Zhang, Xiaotian Lin, Yizhang Zhu, Zhaoyang Yu, Bang Liu, Chenglin Wu, Nan Tang, Yuyu Luo
| Challenge: | Human experts tackle difficult math problems by identifying and executing a few pivotal steps rather than listing every intermediate thought. |
| Approach: | They propose a method for producing training data that mirrors concise human reasoning by rewriting a problem's solution to retain only the essential steps. |
| Outcome: | The proposed method outperforms models trained on 800k long CoT and cuts training and inference costs. |