Papers by Jia Deng
Think Visually: Question Answering through Virtual Imagery (P18-1)
Copied to clipboard
| Challenge: | Existing models of geometric reasoning are based on visual representations of objects and objects, but they are not based in symbols or words. |
| Approach: | They propose a new deep network architecture that specializes in answering questions that admit latent visual representations and learns to generate and reason over such representations. |
| Outcome: | The proposed model can generate and reason over latent visual representations and is validated by two synthetic benchmarks. |
Towards Self-Evolving Agents: Enabling Autonomy through Interactive Experience Refinement (2026.findings-acl)
Copied to clipboard
Cheng Yang, Xuemeng Yang, Licheng Wen, Daocheng Fu, Jianbiao Mei, Rong Wu, Pinlong Cai, Yufan Shen, Nianchen Deng, Jia Xu, Botian Shi, Yu Qiao, Haifeng Li
| Challenge: | Large Language Models struggle with complex, multi-step operational tasks because they remain static during inference and cannot learn from past experience. |
| Approach: | They propose a framework that organizes cross-domain insights to facilitate orchestration of long-horizon workflows. |
| Outcome: | The proposed framework outperforms existing methods on the TAC productivity benchmark and shows strong cross-task transferability. |
I2E: From Image Pixels to Actionable Interactive Environments for Text-Guided Image Editing (2026.acl-long)
Copied to clipboard
Jinghan Yu, Junhao Xiao, Chenyu Zhu, Jiaming Li, Jia Li, HanMing Deng, Xirui Wang, Guoli Jia, Jianjun Li, Xiang Bai, Bowen Zhou, Zhiyuan Ma
| Challenge: | Existing text-guided image editing methods rely on end-to-end pixel-level inpainting paradigm . existing models lack such intermediate representations and Reasoning-then-action process . |
| Approach: | They propose a "Decompose-then-Action" paradigm that revisits image editing as an actionable interaction process within a structured environment. |
| Outcome: | The proposed paradigm outperforms existing methods in compositional editing tasks. |
STYLE: Improving Domain Transferability of Asking Clarification Questions in Large Language Model Powered Conversational Agents (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for addressing ambiguities in conversational search systems are one-size-fits-all and struggle to achieve effective domain transferability. |
| Approach: | They propose a method to provide search engines with strategies regarding when to ask clarification questions in a post-hoc manner. |
| Outcome: | The proposed method improves search performance 10% on four unseen domains. |
Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents (2024.acl-long)
Copied to clipboard
Cheng Qian, Bingxiang He, Zhong Zhuang, Jia Deng, Yujia Qin, Xin Cong, Zhong Zhang, Jie Zhou, Yankai Lin, Zhiyuan Liu, Maosong Sun
| Challenge: | Current language model-driven agents lack mechanisms for effective user participation, which is crucial given the vagueness commonly found in user instructions. |
| Approach: | They propose a benchmark to inspect users’ implicit intentions through explicit queries and a model expert as the upstream in agent design to enhance user-agent interaction. |
| Outcome: | The proposed approach excels at identifying vague user tasks, recovering and summarizing critical missing information, setting precise and necessary agent execution goals, and minimizing redundant tool usage, thus boosting overall efficiency. |
Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for full-attention dLLMs rely on random masking strategies that overlook intrinsic token dependencies. |
| Approach: | They propose an attention-guided denoising and optimization framework that aligns training and optimization with attention-derived dependencies. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on mathematical and coding benchmarks. |
Identifying Visible Actions in Lifestyle Vlogs (P19-1)
Copied to clipboard
| Challenge: | Existing methods for identifying human actions in videos are limited by the number of visual depictions in the videos. |
| Approach: | They propose a multimodal algorithm that leverages visual and linguistic clues to automatically infer which actions are visible in a video. |
| Outcome: | The proposed algorithm can identify actions visible in video while verbally describing them. |
Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond (2025.acl-industry)
Copied to clipboard
Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Tanglifu Tanglifu, Xiaowei Lv, Haosheng Zou, Yongchao Deng, Shousheng Jia, Xiangzheng Zhang
| Challenge: | Experimental results show that opensource curriculum training is more effective when distinct datasets are available for different training stages. |
| Approach: | They propose an opensource suite for training long reasoning models using publicdata and models. |
| Outcome: | The proposed model outperforms DeepSeek-R1-DistillQwen-32B models in math reasoning. |
Task-Adaptive Tokenization: Enhancing Long-Form Text Generation Efficacy in Mental Health and Beyond (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to generate text in mental health are limiting, but they are effective for many tasks. |
| Approach: | They propose a task-adaptive tokenizer that allows for the integration of task-specific tokens into the pre-trained model's tokenization step. |
| Outcome: | The proposed tokenization approach improves generation performance on psychological question-answering tasks in Chinese and English while using 60% fewer tokens. |
Speaker Naming in Movies (N18-1)
Copied to clipboard
| Challenge: | Identifying speakers and their names in movies is a primary task for many video analysis problems, such as automatic subtitle labeling. |
| Approach: | They propose a model that leverages visual, textual, and acoustic modalities in an unified optimization framework for speaker naming in movies. |
| Outcome: | The proposed model outperforms baseline models on the MovieQA 2017 challenge for speaker naming in movies and TV shows on visual, textual, and acoustic modalities. |
Learning How and What to Memorize: Cognition-Inspired Two-Stage Optimization for Evolving Memory (2026.acl-long)
Copied to clipboard
Derong Xu, Shuochen Liu, Pengfei Luo, Pengyue Jia, Yingyi Zhang, Yi Wen, Yimin Deng, Wenlin Zhang, Enhong Chen, Xiangyu Zhao, Tong Xu
| Challenge: | Existing memory systems rely on static, hand-crafted update rules for personalization, but sparse outcome rewards provide weak supervision, resulting in unstable long-horizon optimization. |
| Approach: | They propose a memory guideline optimization framework that learns how memory should be organized and what information to update. |
| Outcome: | The proposed framework learns how memory should be organized and what information to update. |
Strength Lies in Differences! Improving Strategy Planning for Non-collaborative Dialogues via Diversified User Simulation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Non-collaborative dialogue agents are expected to engage in strategic conversations with diverse users, and this poses two main challenges for existing dialogue agents: 1) the inability to integrate user-specific characteristics into the strategic planning; 2) the difficulty of training strategic planners that can be generalized to diverse users. |
| Approach: | They propose to integrate a user-aware strategic planning module and a population-based training paradigm into a non-collaborative dialogue agent for securing a mutual agreement that leans favorably towards the system's objectives. |
| Outcome: | The proposed model can be used to achieve a mutual agreement that leans favorably towards the system's objectives. |
YuLan-Mini: Pushing the Limits of Open Data-efficient Language Model (2025.acl-long)
Copied to clipboard
Hu Yiwen, Huatong Song, Jie Chen, Jia Deng, Jiapeng Wang, Kun Zhou, Yutao Zhu, Jinhao Jiang, Zican Dong, Yang Lu, Xu Miao, Xin Zhao, Ji-Rong Wen
| Challenge: | prevailing pre-training approaches for large language models involve several complexities. |
| Approach: | They propose a low-cost training recipe and a robust optimization approach to mitigate training instability . they also propose synthesis, curriculum, and data selection pipelines to integrate data . |
| Outcome: | The proposed model achieves top-tier performance among models with similar parameter scale . it is comparable to industry-leading models that require significantly more data . |
ValCAT: Variable-Length Contextualized Adversarial Transformations Using Encoder-Decoder Language Model (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing word-level approaches to attack text are limited to a single word . existing methods ignore interactions between consecutive words, resulting in one-to-one attacks . |
| Approach: | They propose a black-box attack framework that misleads the language model by applying variable-length contextualized transformations to the original text. |
| Outcome: | The proposed framework outperforms existing methods on classification and inference tasks. |
The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning (2025.findings-acl)
Copied to clipboard
Bingxiang He, Ning Ding, Cheng Qian, Jia Deng, Ganqu Cui, Lifan Yuan, Haiwen Hong, Huan-ang Gao, Longtao Huang, Hui Xue, Huimin Chen, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing work on instruction tuning has focused on task level, without considering that tasks are artificially defined and, to LLMs, merely consist of tokens and representations. |
| Approach: | They propose a training data arrangement framework that allows for continual learning and loss reduction. |
| Outcome: | The proposed framework promotes continual learning and loss reduction on unseen tasks. |
Can LLMs Act as Historians? Evaluating Historical Research Capabilities of LLMs via the Chinese Imperial Examination (2026.acl-long)
Copied to clipboard
Lirong Gao, Zeqing Wang, Yuyan Cai, Jiayi Deng, Yanmei Gu, Yiming Zhang, Jia Zhou, Yanfei Zhang, Junbo Zhao
| Challenge: | Existing benchmarks assess basic knowledge breadth or lexical understanding, failing to capture higher-order skills that are central to historical research. |
| Approach: | They propose a benchmark anchored in the Chinese Imperial Examination system that assesses historical knowledge and lexical understanding. |
| Outcome: | The new benchmark aims to assess the ability of LLMs to process historical materials and documents. |
MultiDx: A Multi-Source Knowledge Integration Framework towards Diagnostic Reasoning (2026.findings-acl)
Copied to clipboard
Yimin Deng, Zhenxi Lin, Yejing Wang, Guoshuai Zhao, Pengyue Jia, Zichuan Fu, Derong Xu, Yefeng Zheng, Xiangyu Zhao, Li Zhu, Xian Wu, Xueming Qian
| Challenge: | Existing approaches focus on diagnostic reasoning based on internal model knowledge or static knowledge bases. |
| Approach: | They propose a two-stage diagnostic reasoning framework that integrates multi-perspective evidence to generate a diagnostic prediction. |
| Outcome: | The proposed method generates suspected diagnoses and reasoning traces from web search, SOAP-formatted case, and clinical case database. |
Generating Natural Language Proofs with Verifier-Guided Search (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing stepwise methods struggle to generate valid proof steps based on the hypothesis . instead, they generate invalid steps . |
| Approach: | They propose a stepwise method which generates relevant steps conditioning on the hypothesis. |
| Outcome: | The proposed method improves correctness of predicted proofs from 27.7% to 33.3% on EntailmentBank and RuleTaker. |
SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearning (2025.findings-acl)
Copied to clipboard
Junkai Chen, Zhijie Deng, Kening Zheng, Yibo Yan, Shuliang Liu, PeiJun Wu, Peijie Jiang, Jia Liu, Xuming Hu
| Challenge: | Existing methods for MU forget quality and model utility are not fully explored for safety in MLLMs. |
| Approach: | They propose a safety unlearning benchmark for MLLMs to measure over-forgetting . they propose MU methods to forget quality and model utility . |
| Outcome: | The proposed method reduces over-forgetting by 79.5% while maintaining forget quality and model utility. |
LifeQA: A Real-life Dataset for Video Question Answering (2020.lrec-1)
Copied to clipboard
Santiago Castro, Mahmoud Azab, Jonathan Stroud, Cristina Noujaim, Ruoyao Wang, Jia Deng, Rada Mihalcea
| Challenge: | Existing video question answering datasets consist of movies and TV shows, but they are not representative of our day-to-day lives. |
| Approach: | They propose a benchmark dataset for video question answering that focuses on day-to-day situations. |
| Outcome: | The proposed dataset analyzes the challenging but realistic aspects of LifeQA . it consists of video clips and over 2.3k multiple-choice questions . |
SimpleDeepSearcher: Deep Information Seeking via Web-Powered Reasoning Trajectory Synthesis (2025.findings-emnlp)
Copied to clipboard
Shuang Sun, Huatong Song, Yuhao Wang, Ruiyang Ren, Jinhao Jiang, Junjie Zhang, Fei Bai, Jia Deng, Xin Zhao, Zheng Liu, Lei Fang, Zhongyuan Wang, Ji-Rong Wen
| Challenge: | Existing approaches to deep search training lack high-quality training trajectories, prohibitive computational costs and lack of high-fidelity training data. |
| Approach: | They propose a framework that synthesizes high-quality training data by simulating real user interactions in live web search environments. |
| Outcome: | The proposed framework synthesizes high-quality training data by simulating user interactions in live web search environments. |
Large Language Models Meet Harry Potter: A Dataset for Aligning Dialogue Agents with Characters (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that can create open-domain dialogue agents lack character representation and annotations. |
| Approach: | They propose a dataset to study character alignment and character representation . it includes all dialogue sessions from the Harry Potter series and includes annotations . |
| Outcome: | The proposed dataset can be used as a universal benchmark for character-driven LLMs. |
Hi-ToM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Theory of Mind (ToM) is the ability to reason about one's own and others' mental states. |
| Approach: | They propose a higher-order theory of mind benchmark and introduce a new deception mechanism to evaluate ToM reasoning. |
| Outcome: | The proposed benchmarks show that the LLMs are not performing well on higher-order tasks. |