Papers by Yang You
DentalGPT: Incentivizing Multimodal Reasoning in Dentistry (2026.findings-acl)
Copied to clipboard
Zhenyang Cai, Jiaming Zhang, Junjie Zhao, Ziyi Zeng, Yanchao Li, Liang Jingyi, Junying Chen, Yunjin Yang, Jiajun You, Shuzhi Deng, null Xieruiqiii, Yuanting Chen, Xiangyi Feng, Jianquan Li, Liangyi Chen, Junwen Wang, Shan Jiang, Benyou Wang
| Challenge: | Current multimodal large language models (MLLMs) show limited understanding of dental images. |
| Approach: | They propose a dental-specialized multimodal large language model trained via staged multimodal alignment and reinforcement learning. |
| Outcome: | The proposed model outperforms state-of-the-art models on disease classification and dental VQA tasks. |
Writing by Memorizing: Hierarchical Retrieval-based Medical Report Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for medical image analysis use predefined template databases or ignore hierarchical nature of medical report generation. |
| Approach: | They propose a hierarchical retrieval mechanism to extract both report and sentence-level templates for clinically accurate report generation. |
| Outcome: | The proposed model extracts both report and sentence-level templates for clinically accurate report generation. |
GroupToM-Bench: Benchmarking Group Theory of Mind and Nonlinear Social Emergence in MLLMs (2026.acl-long)
Copied to clipboard
Weidong Tang, Jierui Li, Yueling Hou, Zihan Mei, Can Zhang, Xinyan Wan, Zhiyuan Liang, Pengfei Zhou, Yang You, Wangbo Zhao
| Challenge: | Existing models for general intelligence fail to model how mental states interact and crystallize into group-level outcomes. |
| Approach: | They propose a multimodal benchmark for group-level Theory of Mind (ToM) to probe nonlinear collective behavior. |
| Outcome: | The proposed model performs significantly below human levels, exposing blind spots in modeling social structures and nonlinear collective behavior. |
Sequence Parallelism: Long Sequence Training from System Perspective (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work on memory-efficient parallelisms to reduce time and space complexity focuses on reducing time and complexity from system perspective. |
| Approach: | They propose a memory-efficient parallelism to reduce time and space complexity . they split input sequence into multiple chunks and feed each chunk into GPU . |
| Outcome: | The proposed approach is compatible with most existing parallelisms and makes 4D parallelismal possible. |
Are AI-Generated Text Detectors Robust to Adversarial Perturbations? (2024.acl-long)
Copied to clipboard
| Challenge: | Existing detectors for AI-generated text lack robustness against adversarial perturbations, with even minor changes in characters or words causing a reversal in distinguishing between human-created and AI-generated text. |
| Approach: | They propose a siamese calibration technique to train the model to make equally confident predictions under different noise, which improves the model’s robustness against adversarial perturbations. |
| Outcome: | The proposed detector outperforms baseline methods on four datasets and is more generalizable in cross-domain, cross-genre, and mixed-source scenarios. |
MReD: A Meta-Review Dataset for Structure-Controllable Text Generation (2022.findings-acl)
Copied to clipboard
| Challenge: | a new text generation dataset is needed to controllable text summarization, but it lacks the domain knowledge. |
| Approach: | They propose to use existing text generation datasets to leverage input and control signals . they propose to annotate each meta-review sentence manually with a control signal . |
| Outcome: | The proposed method can be used to control the structure of a text generation dataset . it can be applied to a variety of tasks, including a task with a large number of meta-review sentences . |
Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | ChatGPT and GPT-4 are popular as evaluation metric for complex generative tasks . however, they are not ready as human replacements due to significant limitations . |
| Approach: | They conduct extensive analysis to examine the stability and reliability of LLMs as automatic evaluators for abstractive summarization. |
| Outcome: | The proposed methods outperform the commonly used automatic metrics but are not ready for human evaluation due to significant limitations. |
IEvoAgent: Evolving Conversational Agent based on User Implicit Feedback (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to optimize conversational agents often rely on explicit preference pairs and expert evaluations. |
| Approach: | They propose a conversational agent framework that leverages the structured dependency between agent responses and user reactions to extract implicit feedback. |
| Outcome: | The proposed framework improves on MT-Bench-101, WildBench, and FB-Bech, and shows that mining implicit feedback supports better multi-turn alignment under evolving user preferences. |
ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Moments (2026.acl-long)
Copied to clipboard
| Challenge: | Existing defenses for Large Reasoning Models (LRMs) depend on costly fine-tuning and additional expert knowledge, which limits their scalability. |
| Approach: | They propose an inference-time safeguard for Large Reasoning Models that injects safety aha moments into the reasoning process to guide the model towards harmless yet helpful reasoning. |
| Outcome: | The proposed safeguard outperforms nine existing safeguards while avoiding common exaggerated safety issues. |
Multimodal Prompt Learning for Product Title Generation with Extremely Limited Labels (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to generate informative titles for products with limited labels are inadequate for novel products. |
| Approach: | They propose a prompt-based approach to generate attractive titles for novel products . they use multimodal prompts to preserve characteristics and writing styles of novel products. |
| Outcome: | The proposed approach achieves state-of-the-art results on novel product categories with limited labels. |
InfiMM: Advancing Multimodal Understanding with an Open-Sourced Visual Language Model (2024.findings-acl)
Copied to clipboard
Haogeng Liu, Quanzeng You, Yiqi Wang, Xiaotian Han, Bohan Zhai, Yongfei Liu, Wentao Chen, Yiren Jian, Yunzhe Tao, Jianbo Yuan, Ran He, Hongxia Yang
| Challenge: | InfiMM is a multimodal large language model that adapts to complex vision-language tasks. |
| Approach: | They present a Multimodal Large Language Model that adapts to intricate vision-language tasks using large-scale training data and comprehensive training strategies. |
| Outcome: | Empirical evaluations across a variety of benchmarks underscore InfiMM’s remarkable capability in multimodal understanding. |
Improving Abstractive Document Summarization with Salient Information Modeling (P19-1)
Copied to clipboard
| Challenge: | Abstractive document summarization is a task of natural language generation which generates fluent summaries with salient information automatically. |
| Approach: | They propose to incorporate a Gaussian focal bias on attention scores into an encoder to enhance the perception of local context and to distinguish salient information precisely. |
| Outcome: | The proposed framework outperforms state-of-the-art models on the CNN/Daily Mail benchmark and is based on a focus-attention mechanism and two new extensions. |
SentBS: Sentence-level Beam Search for Controllable Summarization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Structure-controlled summarization is a useful and interesting research direction . current structure-controlling methods have limited effectiveness in enforcing the desired structure. |
| Approach: | They propose a sentence-level beam search generation method to select suitable sentences for subsequent generations. |
| Outcome: | The proposed method significantly reduces structural discrepancies by 68% on a structure-controlled dataset. |
Causal Reasoning of Entities and Events in Procedural Texts (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing work on entity state tracking or event reasoning is limited to procedural texts. |
| Approach: | They propose a benchmark for causal reasoning of event plausibility and entity states . they represent entities as programming languages while prompting language models . |
| Outcome: | The proposed model outperforms existing models on human reasoning and event reasoning. |
MultiAgentBench : Evaluating the Collaboration and Competition of LLM agents (2025.acl-long)
Copied to clipboard
Kunlun Zhu, Hongyi Du, Zhaochen Hong, Xiaocheng Yang, Shuyi Guo, Zhe Wang, Zhenhailong Wang, Cheng Qian, Robert Tang, Heng Ji, Jiaxuan You
| Challenge: | Large Language Models (LLMs) have shown remarkable capabilities as autonomous agents, yet existing benchmarks focus on single-agent tasks or are confined to narrow domains, failing to capture the dynamics of multi-agent coordination and competition. |
| Approach: | They propose a benchmark to evaluate LLM-based multi-agent systems across diverse, interactive scenarios. |
| Outcome: | The proposed framework measures task completion and quality of collaboration and competition using novel, milestone-based key performance indicators. |
JTAV: Jointly Learning Social Media Content Representation by Fusing Textual, Acoustic, and Visual Features (C18-1)
Copied to clipboard
| Challenge: | Existing studies on learning social media content focus on single modal or bi-modal learning, but this approach is non-trivial and challenging because content is multi-modal and involves several types of data, including text, audio, and image. |
| Approach: | They propose to combine textual, acoustic, and visual information to learn social media content by fusing them jointly. |
| Outcome: | The proposed model outperforms the state-of-the-art approaches on real-world datasets by a large margin. |
InfiMM-WebMath-40B: Advancing Multimodal Pre-Training for Enhanced Mathematical Reasoning (2025.findings-emnlp)
Copied to clipboard
Xiaotian Han, Yiren Jian, Xuefeng Hu, Haogeng Liu, Yiqi Wang, Qihang Fan, Yuang Ai, Huaibo Huang, Ran He, Zhenheng Yang, Quanzeng You
| Challenge: | InfiMM-WebMath-40B is a dataset of interleaved image-text documents . it consists of 24 million web pages, 85 million image URLs, and 40 billion text tokens . |
| Approach: | InfiMM-WebMath-40B is a high-quality dataset of interleaved image-text documents . it contains 24 million web pages, 85 million image URLs, and 40 billion text tokens . |
| Outcome: | InfiMM-WebMath-40B is a high-quality dataset of interleaved image-text documents . it consists of 24 million web pages, 85 million image URLs, and 40 billion text tokens . |
Memory-Driven Role-Playing: Evaluation and Enhancement of Persona Knowledge Utilization in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing models fail to recall and accurately apply designated persona knowledge without explicit cues . memory-driven role-playing paradigms are attracting significant interest . |
| Approach: | They propose a memory-driven role-playing paradigm that frames persona knowledge as the LLM's internal memory store and a prompting architecture that guides structured memory retrieval and response generation. |
| Outcome: | The proposed paradigm provides a comprehensive diagnostic for four-stage role-playing abilities across 12 LLMs. |
Evaluating the Expressive Appropriateness of Speech in Rich Contexts (2026.acl-long)
Copied to clipboard
Tianrui Wang, Ziyang Ma, Yizhou Peng, Haoyu Wang, Zhikang Niu, Zikang Huang, Yihao Wu, Yi-Wen Chao, Yu Jiang, Yuheng Lu, Guanrou Yang, Xuanchen Li, Hexin Liu, Chunyu Qiang, Cheng Gong, Yifan Yang, Tianchi Liu, Junyu Wang, Nana Hou, Meng Ge, Fuming You, Yang Wei, Zhongqian Sun, Hu Haifeng, Xiaobao Wang, Eng Siong Chng, Xie Chen, Longbiao Wang, Jianwu Dang
| Challenge: | Existing methods for evaluating expressive speech focus on word accuracy, naturalness, signal quality, or emotional intensity at the utterance level. |
| Approach: | They propose a framework for Evaluating Expressive Appropriateness in speech that assesses whether a speech sample aligns with the underlying communicative intent implied by its discourse-level narrative context. |
| Outcome: | The proposed framework outperforms existing speech evaluation and analysis systems on a human-annotated test set. |
How Does the Textual Information Affect the Retrieval of Multimodal In-Context Learning? (2024.emnlp-main)
Copied to clipboard
| Challenge: | MLLMs have significant capabilities for multimodal in-context learning, but their effectiveness hinges on the appropriate selection of in-constext examples. |
| Approach: | They propose a supervised MLLM prompt retriever that leverages a trained retriever based on MLML's confidence to select examples, which enhances multimodal in-context learning efficiency. |
| Outcome: | The proposed method is validated through extensive testing across three different tasks and demonstrates its effectiveness. |
Multiplex Graph Neural Network for Extractive Text Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for extractive text summarization do not consider multiple types of inter-sentential relationships, nor model intra-sententential relationships. |
| Approach: | They propose a novel method to combine different types of relationships among sentences and words to model sentence embedding. |
| Outcome: | The proposed model is compared with existing methods on CNN/DailyMail benchmark dataset to demonstrate its effectiveness. |
CAME: Confidence-guided Adaptive Memory Efficient Optimization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing memory-efficient methods require second-moment estimates of the per-parameter gradients to maintain their performance. |
| Approach: | They propose to use memory-efficient optimizers to reduce memory usage by preserving second-moment estimates of gradients. |
| Outcome: | The proposed method achieves fast convergence and lower memory usage across training tasks. |
Bridging the Gap between Training and Inference for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation generates target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Approach: | They propose to use ground truth and inference to generate target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Outcome: | Experiments on Chinese->English and WMT’14 English->German translation tasks show that the proposed model can achieve significant improvements on multiple datasets. |
POINTS-Reader: Distillation-Free Adaptation of Vision-Language Models for Document Conversion (2025.emnlp-main)
Copied to clipboard
Yuan Liu, Zhongyin Zhao, Le Tian, Haicheng Wang, Xubing Ye, Yangxiu You, Zilin Yu, Chuhan Wu, Zhou Xiao, Yang Yu, Jie Zhou
| Challenge: | Existing approaches to training document conversion models with manual annotation are costly and time-consuming, and training student models by distilling outputs from teacher models can significantly limit their performance in real-world applications. |
| Approach: | They propose a fully automated framework for constructing high-quality document extraction datasets and models capable of handling diverse document formats and layouts. |
| Outcome: | The proposed model outperforms existing models and improves on annotated documents. |
A Hierarchical Encoding-Decoding Scheme for Abstractive Multi-document Summarization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models have been used for abstractive single-document summarization (SDS) but they may not be suitable for multi-document summary (MDS) |
| Approach: | They propose to enforce hierarchy on both encoder and decoder to facilitate multi-document interactions for MDS. |
| Outcome: | Xiao et al. (2019) outperforms or is competitive with the previous best models. |
DavIR: Data Selection via Implicit Reward for Large Language Models (2025.acl-long)
Copied to clipboard
Haotian Zhou, Tingkai Liu, Qianli Ma, Yufeng Zhang, Jianbo Yuan, Pengfei Liu, Yang You, Hongxia Yang
| Challenge: | 6% of Alpaca dataset selected with DavIR can steer both LLaMA and Gemma models to produce superior performance compared to the same models trained on the full 52K dataset. |
| Approach: | They propose a model-based data selection method for post-training Large Language Models . they generalize Reducible Holdout Loss to core-set selection problem of causal language modeling . |
| Outcome: | The proposed method can steer both LLaMA and Gemma models to superior performance compared to the same models trained on the full 52K dataset. |