Papers by Shen Yan

Transductive Learning for Unsupervised Text Style Transfer (2021.emnlp-main)

Challenge:	Current scientific reasoning models struggle with generalization across domains and fall short of multimodal perception.
Approach:	They propose to use multimodal large language models to integrate text, images, and other modalities to enhance scientific reasoning.
Outcome:	The proposed models can integrate text, images, and other modalities and improve reasoning across disciplines.

Copied to clipboard

Fei Xiao, Liang Pang, Yanyan Lan, Yan Wang, Huawei Shen, Xueqi Cheng

Word Graph Guided Summarization for Radiology Findings (2021.findings-acl)

Challenge:	Existing methods for style transfer are based on an inductive learning approach, which represents the style as embeddings, decoder parameters, or discriminator parameters and directly applies these general rules to the test cases.
Approach:	They propose a retrieval-based context-aware style representation that involves top-K relevant sentences in the target style in the transfer process.
Outcome:	The proposed method outperforms several strong baselines and is general and effective to the task of unsupervised style transfer.

Copied to clipboard

Jinpeng Hu, Jianling Li, Zhihong Chen, Yaling Shen, Yan Song, Xiang Wan, Tsung-Hui Chang

A Deep Learning-Based System for PharmaCoNER (D19-57)

Challenge:	Existing studies focus on introducing salient word information to general text summarization framework to guide selection of key content in radiology findings.
Approach:	They propose a method for automatic impression generation using word graphs and a Word Graph guided Summarization model to capture critical words and their relations.
Outcome:	The proposed method is validated on two datasets, OPENI and MIMIC-CXR.

Copied to clipboard

Ying Xiong, Yedan Shen, Yuanhang Huang, Shuai Chen, Buzhou Tang, Xiaolong Wang, Qingcai Chen, Jun Yan, Yi Zhou

Information Retrieval

Challenge:	Efficient access to mentions of clinical entities is very important for using clinical text.
Approach:	They developed a pipeline system based on deep learning methods for this shared task . it achieves a micro-average F1-score of 0.9105 on track 1 and a mini-average LSTM score of 0.8391 on track 2 .
Outcome:	The proposed system achieves a micro-average F1-score of 0.9105 on track 1 and a mini-average score of 0.8391 on track 2.

LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin (2024.acl-long)

Copied to clipboard

Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Wei Shen, Limao Xiong, Yuhao Zhou, Xiao Wang, Zhiheng Xi, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang

Experience-driven Multi-turn Reinforcement Learning for GUI Agents (2026.acl-long)

Challenge:	Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM.
Approach:	They propose a framework that introduces several low-rank adapters and integrates them by using a router network to freeze the backbone model and force a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks.
Outcome:	The proposed framework freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge forgetting.

Copied to clipboard

Zhengxi Lu, Jiabo Ye, Fei Tang, Yongliang Shen, Haiyang Xu, Ziwei Zheng, Weiming Lu, Ming Yan, Fei Huang, Jun Xiao, Yueting Zhuang

Challenge:	GUI agents have demonstrated remarkable progress in automating complex user interface interactions . training such agents for long-horizon tasks remains challenging due to limited rewards and prohibitive costs.
Approach:	They propose a method that leverages expert trajectories as environment experiences for on-policy multi-turn training.
Outcome:	The proposed method achieves significant gains over the base model with 1K public trajectories as RL experiences . it achieves competitive performance against strong baselines such as UI-TARS-7B and GPT-4o .

Capturing Relations between Scientific Papers: An Abstractive Model for Related Work Section Generation (2021.acl-long)

Copied to clipboard

Xiuying Chen, Hind Alamro, Mingzhe Li, Shen Gao, Xiangliang Zhang, Dongyan Zhao, Rui Yan

Structured Data in NLP

Challenge:	Existing related work generation models are inflexible and extract sentences from multiple papers to form a related work discussion.
Approach:	They propose a Relation-aware Related work generator which generates an abstractive related work from the given multiple scientific papers in the same research area.
Outcome:	The proposed model improves over existing models and can be used to familiarize researchers with the state of the art in the field.

The Illusion of Specialization: Unveiling the Domain-Invariant "Standing Committee" in Mixture-of-Experts Models (2026.acl-long)

Copied to clipboard

Yan Wang, Yitao Xu, Nanhan Shen, Jinyan Su, Jimin Huang, Zining Zhu

Challenge:	Mixture of Experts models are widely assumed to achieve domain specialization through sparse routing.
Approach:	They propose a framework that analyzes routing behavior at the level of expert groups rather than individual experts.
Outcome:	The proposed framework analyzes routing behavior at the level of expert groups rather than individual experts.

SocialBench: Sociality Evaluation of Role-Playing Conversational Agents (2024.findings-acl)

Copied to clipboard

Hongzhan Chen, Hehong Chen, Ming Yan, Wenshen Xu, Gao Xing, Weizhou Shen, Xiaojun Quan, Chenliang Li, Ji Zhang, Fei Huang

CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation (2025.emnlp-main)

Challenge:	Existing studies on role-playing agents have focused on enhancing their conversational capability, role-specific knowledge and style, but there has been a gap in assessing their social intelligence.
Approach:	They propose a benchmark to evaluate the sociality of role-playing agents using LLMs.
Outcome:	The proposed benchmark is constructed from various sources and covers a wide range of 500 characters and over 6,000 question prompts and 30,800 multi-turn role-playing utterances.

Copied to clipboard

Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, Yulan He

Iterative Document Representation Learning Towards Summarization with Polishing (D18-1)

Challenge:	Prior implicit CoT methods have underperformed in terms of efficiency and robustness by relying on natural language tokens for reasoning.
Approach:	They propose a training framework that compresses natural language CoT into continuous space by aligning hidden states of a designated token.
Outcome:	The proposed framework outperforms the existing state-of-the-art in 3.1x compression rate and 28.2% accuracy on GSM8k scale.

Copied to clipboard

Xiuying Chen, Shen Gao, Chongyang Tao, Yan Song, Dongyan Zhao, Rui Yan

TILGAN: Transformer-based Implicit Latent GAN for Diverse and Coherent Text Generation (2021.findings-acl)

Challenge:	Existing summarization methods read through document only once to generate a document representation, resulting in a sub-optimal representation.
Approach:	They propose an iterative model for supervised extractive text summarization which polishes the document representation on many passes through the document.
Outcome:	The proposed model outperforms state-of-the-art extractive systems on CNN/DailyMail and DUC2002 datasets.

Copied to clipboard

Shizhe Diao, Xinwei Shen, Kashun Shum, Yan Song, Tong Zhang

Pause or Fabricate? Training Language Models for Grounded Reasoning (2026.findings-acl)

Challenge:	Existing autoregressive models suffer from the exposure bias problem due to mismatches between training and generation stages.
Approach:	They propose a Transformerbased Implicit Latent GAN which combines a transformer autoencoder and GAN in the latent space with a novel design and distribution matching based on the Kullback-Leibler divergence.
Outcome:	The proposed model improves local and global coherence and quality-diversity trade-off on three benchmark datasets.

Copied to clipboard

Yiwen Qiu, Linjuan Wu, Yizhou Liu, Yuchen Yan, Jin Ma, Xu Tan, Yao Hu, Daoxin Zhang, Wenqi Zhang, Weiming Lu, Jun Xiao, Yongliang Shen

Writing-RL: Advancing Long-form Writing via Adaptive Curriculum Reinforcement Learning (2026.acl-long)

Challenge:	Large language models implicitly fabricate information when inputs are incomplete, causing confidence but unreliable conclusions.
Approach:	They propose a framework for grounded reasoning under incomplete information that decomposes reasoning into two stages . they propose stage-specific rewards to penalize hallucinations, enabling models to detect gaps, stop proactively, and resume reasoning after clarification.
Outcome:	The proposed framework improves premise detection and task success by 30% . it also reduces average response length by over 20% .

Copied to clipboard

Xuanyu Lei, Chenliang Li, Yuning Wu, Kaiming Liu, Weizhou Shen, Peng Li, Ming Yan, Fei Huang, Ya-Qin Zhang, Yang Liu

Challenge:	Recent advances in Large Language Models (LLMs) have enabled strong performance in long-form writing, but current training paradigms remain limited.
Approach:	They propose an Adaptive Curriculum Reinforcement Learning framework to advance long-form writing capabilities beyond SFT.
Outcome:	Experiments on 7B-scale writer models show that Writing-RL improves long-form writing performance over strong SFT baselines.

A Systematic Survey of Automatic Prompt Optimization Techniques (2025.emnlp-main)

Copied to clipboard

Kiran Ramnath, Kang Zhou, Sheng Guan, Soumya Smruti Mishra, Xuan Qi, Zhengyuan Shen, Shuai Wang, Sangmin Woo, Sullam Jeoung, Yawei Wang, Haozhu Wang, Han Ding, Yuzhe Lu, Zhichao Xu, Yun Zhou, Balasubramaniam Srinivasan, Qiaojing Yan, Yueyan Chen, Haibo Ding, Panpan Xu, Lin Lee Cheong

Mitigating Posterior Salience Attenuation in Long-Context LLMs with Positional Contrastive Decoding (2025.acl-short)

Challenge:	Recent advances in prompt engineering have created impediments for end users to adopt . however, prompt engineering remains an impedance due to rapid advances in models, tasks, and associated best practices.
Approach:	They propose to define APO as a 5-part unifying framework and categorize all relevant works based on their salient features.
Outcome:	The proposed framework aims to improve the performance of large language models on various tasks.

Copied to clipboard

Zikai Xiao, Ziyang Wang, Wen Ma, Yan Zhang, Wei Shen, WangYan WangYan, Luqi Gong, Zuozhu Liu

Challenge:	Current solutions incur prohibitive training costs, leaving statistical behaviors and cost-effective approaches underexplored.
Approach:	They propose a positional contrast decoding technique that contrasts long-aware attention with designed local-awn attention.
Outcome:	The proposed model achieves state-of-the-art performance on long-context benchmarks.

Mutual-Taught for Co-adapting Policy and Reward Models (2025.acl-long)

Copied to clipboard

Tianyuan Shi, Canbin Huang, Fanqi Wan, Longguang Zhong, Ziyi Yang, Weizhou Shen, Xiaojun Quan, Ming Yan

Machine Translation

Challenge:	Experimental results show that this iterative approach leads to consistent improvements in both the policy model and reward model.
Approach:	They propose a method that iteratively improves both the policy model and reward model without requiring additional human annotation.
Outcome:	The proposed method improves both the policy model and reward model without human annotation.

Selection and Generation: Learning towards Multi-Product Advertisement Post Generation (2020.emnlp-main)

Copied to clipboard

Zhangming Chan, Yuchi Zhang, Xiuying Chen, Shen Gao, Zhiqiang Zhang, Dongyan Zhao, Rui Yan

Text Generation

Challenge:	E-commerce websites have billions of products, so it is impossible to write all copywriting manually.
Approach:	They propose a model to generate an AD post using a select network and a MGenNet network to generate a post including selected products.
Outcome:	The proposed model achieves impressive performance on a large-scale real-world AD post dataset.

Amalgamating Knowledge from Two Teachers for Task-oriented Dialogue System with Adversarial Training (2020.emnlp-main)

Copied to clipboard

Wanwei He, Min Yang, Rui Yan, Chengming Li, Ying Shen, Ruifeng Xu

Robustness in NLP

Challenge:	Existing approaches to task-oriented dialogue systems require a large number of handcrafted features and labels.
Approach:	They propose a "Two-Teacher One-Student" learning framework for task-oriented dialogue . the framework amalgamates knowledge from two teacher networks and provides guidance .
Outcome:	The proposed framework outperforms baseline methods on two benchmark datasets . it can retrieve accurate KB entities and generate human-like responses simultaneously .

Scientific Paper Extractive Summarization Enhanced by Citation Graphs (2022.emnlp-main)

Copied to clipboard

Xiuying Chen, Mingzhe Li, Shen Gao, Rui Yan, Xin Gao, Xiangliang Zhang

LiGen: Active Lipid Generation via a Molecular Language Model (2026.acl-long)

Challenge:	citation graphs can be used to extract scientific papers under different conditions.
Approach:	They propose a multi-granularity unsupervised summarization model that fine tunes a pre-trained encoder model on the citation graph by link prediction tasks.
Outcome:	The proposed model outperforms baseline models on a public benchmark dataset.

Copied to clipboard

Ying Zhan, Xiuqi Tang, Yan Zhang, Xiao Tan, Dian Shen, Zhou Yu, Beilun Wang

Learning to Use Tools via Cooperative and Interactive Agents (2024.findings-emnlp)

Challenge:	Lipid nanoparticles (LNPs) can deliver cargos to tumor and immune cells . traditional approaches rely on experimental screening and expert judgment .
Approach:	They propose a method to generate lipid molecules efficiently and actively using deep learning.
Outcome:	The proposed method outperforms baseline methods on multiple cell lines and achieves a 30% improvement over the current methods.

Copied to clipboard

Zhengliang Shi, Shen Gao, Xiuyi Chen, Yue Feng, Lingyong Yan, Haibo Shi, Dawei Yin, Pengjie Ren, Suzan Verberne, Zhaochun Ren

Green & Sustainable NLP

Challenge:	Existing methods for large language models (LLMs) use one agent to iterate and execute tools, but they suffer from performance degradation when addressing practical tasks.
Approach:	They propose a tool learning framework that coordinates three specialized agents for tool selection, tool execution, and action calibration separately.
Outcome:	The proposed framework outperforms baseline models on three datasets with 14% higher success rate.

LLMs for Mathematical Modeling: Towards Bridging the Gap between Natural and Mathematical Languages (2025.findings-naacl)

Copied to clipboard

Xuhan Huang, Qingning Shen, Yan Hu, Anningzhe Gao, Benyou Wang

CESRec: Constructing Pseudo Interactions for Sequential Recommendation via Conversational Feedback (2025.findings-emnlp)

Challenge:	Large Language Models (LLMs) have demonstrated strong performance across various natural language processing tasks, but their proficiency in mathematical reasoning remains a key challenge.
Approach:	They propose a process-oriented framework to evaluate LLMs' ability to construct mathematical models, using solvers to compare outputs with ground truth.
Outcome:	The proposed framework evaluates LLMs' ability to construct mathematical models, using solvers to compare outputs with ground truth.

Copied to clipboard

Yifan Wang, Shen Gao, Jiabao Fang, Rui Yan, Billy Chiu, Shuo Shang

ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detection (2026.findings-acl)

Challenge:	Existing Sequential Recommendation Systems (SRS) rely on collaborative filtering signals and fail to capture real-time user preferences.
Approach:	They propose a framework that integrates the long-term preference modeling of SRS with the real-time preference elicitation of CRS.
Outcome:	The proposed framework integrates the long-term preference modeling of SRS with the real-time preference elicitation of CRS.

Copied to clipboard

Yibo Yan, Shen Wang, Jiahao Huo, Hang Li, Boyan Li, Jiamin Su, Xiong Gao, YiFan Zhang, Tianlong Xu, Zhendong Chu, Aoxiao Zhong, Kun Wang, Hui Xiong, Philip S. Yu, Xuming Hu, Qingsong Wen

Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow (2026.findings-acl)

Challenge:	Current mathematical benchmarks focus on evaluating MLLMs’ problem-solving ability, yet there is a crucial gap in addressing more complex scenarios such as error detection.
Approach:	They propose to evaluate multimodal error detection by evaluating two sub-tasks error step identification and error categorization.
Outcome:	The proposed task evaluates MLLMs' ability to handle multimodal questions compared to text-only models.

Copied to clipboard

Yangyang Zhong, Yanmei Gu, Zhengqing Zang, Xiaomeng Li, Yuqi Ding, Xibei Jia, Yuting Shen, Zhenzhong Lan, Liwang Zhu, Weiping Liu, Junlin Zhou, Haisheng Liu, Zhong Xin Yu, Pengxin Luo, Donglian Qi, Yunfeng Yan, Junbo Zhao

BioGen: Generating Biography Summary under Table Guidance on Wikipedia (2021.findings-acl)

Challenge:	Autoregressive (AR) language models dominate modern natural language processing due to strong likelihood-based training objectives and reliable left-to-right decoding.
Approach:	They characterize MDLM behavior along two dimensions: parallelism strength and generation order . authors propose a Generate-then-Edit paradigm that mitigates dependency loss .
Outcome:	The proposed model improves on tasks that require "backward information" the Generate-then-Edit paradigm improves parallel decoding efficiency while reducing dependency loss.

Copied to clipboard

Shen Gao, Xiuying Chen, Chang Liu, Dongyan Zhao, Rui Yan

Predicting Text Preference Via Structured Comparative Reasoning (2024.acl-long)

Challenge:	Existing methods for summarizing text have not captured the salient information from an article.
Approach:	They propose a table-guided abstractive biography summarization that utilizes factual tables to capture important information and generate a summary of a biography.
Outcome:	The proposed method is the first large-scale biography summarization dataset with tables.

Copied to clipboard

Jing Nathan Yan, Tianqi Liu, Justin Chiu, Jiaming Shen, Zhen Qin, Yue Yu, Charumathi Lakshmanan, Yair Kurzion, Alexander Rush, Jialu Liu, Michael Bendersky

SalaMAnder: Shapley-based Mathematical Expression Attribution and Metric for Chain-of-Thought Reasoning (2025.findings-emnlp)

Challenge:	Existing approaches to comparative reasoning rely on pretraining or fine-tuning models at the cost of massive human annotation and computation.
Approach:	They propose a model that prompts LLMs to generate structured intermediate comparisons by proposing aspects for comparison, followed by generating textual comparisons under each aspect.
Outcome:	The proposed model significantly reduces hallucination and improves consistency across various NLP tasks.

Copied to clipboard

Yue Xin, Chen Shen, Shaotian Yan, Xiaosong Yuan, Yaoming Wang, Xiaofeng Zhang, Chenxi Huang, Jieping Ye

TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities (2023.acl-demo)

Challenge:	Chain-of-Thought prompting improves the math reasoning capability of large language models.
Approach:	They propose a method for attribution of component-level contributions in CoT reasoning using Shapley value and a stratified sampling algorithm that significantly reduces computational complexity.
Outcome:	The proposed method reduces computational complexity and provides robust correlations with model performance.

Copied to clipboard

Zhe Zhao, Yudong Li, Cheng Hou, Jing Zhao, Rong Tian, Weijie Liu, Yiren Chen, Ningyuan Sun, Haoyan Liu, Weiquan Mao, Han Guo, Weigang Gou, Taiqiang Wu, Tao Zhu, Wenhang Shi, Chen Chen, Shan Huang, Sihong Chen, Liqun Liu, Feifei Li, Xiaoshuai Chen, Xingwu Sun, Zhanhui Kang, Xiaoyong Du, Linlin Shen, Kimmo Yan

Cross-modal Memory Networks for Radiology Report Generation (2021.acl-long)

Challenge:	Several pre-training models of different modalities are showing a rising trend of homogeneity in their model structures.
Approach:	They propose a toolkit that supports pre-training models of different modalities.
Outcome:	The proposed toolkit can match the performance of the original implementations on text, vision, and audio benchmarks.

Copied to clipboard

Zhihong Chen, Yaling Shen, Yan Song, Xiang Wan

Small LLMs Are Weak Tool Learners: A Multi-LLM Agent (2024.emnlp-main)

Challenge:	Medical imaging reports are essential in clinical practice, and generating the reports is beneficial to reduce the burden of radiologists.
Approach:	They propose to use a shared memory to enhance the encoder-decoder framework for radiology report generation.
Outcome:	The proposed model can generate more accurate reports on two widely used datasets.

Copied to clipboard

Weizhou Shen, Chenliang Li, Hongzhan Chen, Ming Yan, Xiaojun Quan, Hehong Chen, Ji Zhang, Fei Huang

RealBench: A Chinese Multi-image Understanding Benchmark Close to Real-world Scenarios (2025.findings-emnlp)

Challenge:	Large Language Models (LLMs) have revolutionized natural language processing with impressive capabilities, but they lack domain specificity, real-time information and face challenges in solving specialized problems.
Approach:	They propose a multi-LLM approach that decomposes the aforementioned capabilities into a planner, caller, and summarizer.
Outcome:	The proposed model outperforms existing models by demonstrating its effectiveness and advantages in tool learning.

Copied to clipboard

Fei Zhao, Chengqiang Lu, Yufan Shen, Qimeng Wang, Yicheng Qian, Haoxin Zhang, Yan Gao, null Yiwu, Yao Hu, Zhen Wu, Shangyu Xing, Xinyu Dai

AskToAct: Enhancing LLMs Tool Use via Self-Correcting Clarification (2025.emnlp-main)

Challenge:	RealBench is the first Chinese multimodal multi-image dataset . the dataset contains 9393 samples and 69910 images .
Approach:	They propose to create a Chinese multimodal multi-image dataset using 21 models . they use closed-source models that support multi-inputs as well as open-source visual and video models a .
Outcome:	The first Chinese multimodal multi-image dataset contains 9393 samples and 69910 images.

Copied to clipboard

Xuan Zhang, Yongliang Shen, Zhe Zheng, Linjuan Wu, Wenqi Zhang, Yuchen Yan, Qiuying Peng, Jun Wang, Weiming Lu

CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolution (2026.acl-long)

Challenge:	Existing tools for ambiguous and incomplete queries are limited by manual construction and lack of error correction mechanisms during multi-turn clarification.
Approach:	They propose a framework that exploits the mapping between queries and their tool invocation solutions by removing key parameters from queries while retaining them as ground truth.
Outcome:	The proposed framework outperforms existing methods while maintaining high accuracy in tool invocation.

Copied to clipboard

Teng Pan, Yuchen Yan, Zixuan Wang, Ruiqing Zhang, Guiyang Hou, Wenqi Zhang, Weiming Lu, Jun Xiao, Yongliang Shen

LLM Agents for Education: Advances and Applications (2025.findings-emnlp)

Challenge:	Label-free reinforcement learning enables large language models to improve reasoning capabilities . but as training maximizes self-consistency, output diversity collapses, authors say . authors propose a framework where a single model alternates between generator and verifier roles .
Approach:	They propose a framework where a model alternates between generator and verifier roles, bootstrapping each other.
Outcome:	Experiments show that CoVerRL outperforms label-free baselines on reasoning benchmarks . the framework can be used to improve reasoning abilities without ground-truth supervision .

Copied to clipboard

Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jingheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S. Yu, Qingsong Wen

System Prompt Hijacking via Permutation Triggers in LLM Supply Chains (2025.findings-acl)

Challenge:	Large Language Model (LLM) agents are transforming education by automating complex tasks and enhancing both teaching and learning processes.
Approach:	This survey analyzes recent advances in applying Large Language Model agents to educational settings . it highlights ethical issues, hallucination and overreliance, and integration with existing ecosystems .
Outcome:	The authors analyze the technologies enabling LLM agents and highlight key challenges in deploying them in educational settings.

Copied to clipboard

Lu Yan, Siyuan Cheng, Xuan Chen, Kaiyuan Zhang, Guangyu Shen, Xiangyu Zhang

CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation (2024.acl-long)

Challenge:	Large language models (LLMs) are rapidly transforming the landscape of artificial intelligence due to the substantial resources required for training.
Approach:	They propose a post-deployment attack that bypasses system prompts to compromise models . they introduce Precise Activation Guarding and Unit Deviation Sampling to protect against attack .
Outcome:	The proposed attack bypasses system prompts, enabling unrestricted model outputs and safety violations.

Copied to clipboard

Quan Tu, Shilong Fan, Zihang Tian, Tianhao Shen, Shuo Shang, Xin Gao, Rui Yan

Summarizing Procedural Text: Data and Approach (2022.findings-emnlp)

Challenge:	CharacterEval is a benchmark for comprehensive RPCA assessment in Chinese . authors show that Chinese LLMs exhibit more promising capabilities than GPT-4 in role-playing conversation.
Approach:	They propose a Chinese benchmark for comprehensive RPCA assessment . they use a dataset of Chinese role-playing dialogues and character profiles .
Outcome:	The proposed benchmark demonstrates that Chinese LLMs exhibit more promising capabilities than GPT-4 in Chinese role-playing conversation.

Copied to clipboard

Shen Gao, Haotong Zhang, Xiuying Chen, Rui Yan, Dongyan Zhao

From Redundancy to Relevance: Information Flow in LVLMs Across Reasoning Tasks (2025.naacl-long)

Challenge:	Procedural text summarization task is a popular task in the NLP field because of its long length and complexity.
Approach:	They propose a procedural text summarization task with two granularity . they propose an Entity-State Graph-based Summarizer (ESGS) which aggregates contextual information for each procedure.
Outcome:	The proposed model can summarize the entire procedural text or give an overview for each step or both . Experiments on two datasets confirm the proposed model's effectiveness.

Copied to clipboard

Xiaofeng Zhang, Yihao Quan, Chen Shen, Xiaosong Yuan, Shaotian Yan, Liang Xie, Wenxiao Wang, Chaochen Gu, Hao Tang, Jieping Ye

GovScape: A Public Multimodal Search System for 70 Million Pages of Government PDFs (2026.acl-demo)

Challenge:	Large visionlanguage models (LVLMs) are a powerful visual-language reasoning tool.
Approach:	They propose to integrate attention analysis with LLaVA-CAM to determine interactions between visual representations.
Outcome:	The proposed approach can be used to determine interactions between visual representations.

Copied to clipboard

Ying-Hsiang Huang, Claire Gong, Shreya Shaji, Alison R Yan, Leslie Harka, Albert Du, Anjali Shubha Gopal, Samuel J Klein, Shannon Zejiang Shen, Mark E. Phillips, Trevor Owens, Kyle Deeds, Benjamin Charles Germain Lee

Information Retrieval

Challenge:	Efforts over the past three decades have produced web archives containing billions of webpage snapshots and petabytes of data.
Approach:	They propose a public search system that supports multimodal searches across 10,015,993 federal government PDFs from the 2020 End of Term crawl.
Outcome:	The proposed system supports multimodal searches across 10,015,993 federal government PDFs from the 2020 End of Term crawl (70,958,487 total PDF pages) significant compute cost for GovScape’s pre-processing pipeline for 10 million PDFs was approximately 1,500, equivalent to 47,000 PDF pages per dollar spent on compute.

Learning to Organize a Bag of Words into Sentences with Neural Networks: An Empirical Study (2021.naacl-main)

Copied to clipboard

Chongyang Tao, Shen Gao, Juntao Li, Yansong Feng, Dongyan Zhao, Rui Yan

Challenge:	Existing approaches to encode natural languages without orders are lacking.
Approach:	They conduct a comprehensive analysis of the ability of neural models to organize sentences from a bag of words under three typical scenarios.
Outcome:	The proposed models can reorder or reconstruct sentences from a bag of words under three typical scenarios.

From Observation to Understanding: Front-Door Adjustments with Uncertainty Calibration for Enhancing Egocentric Reasoning in LVLMs (2025.findings-acl)

Copied to clipboard

Shenshen Li, Wenxin Meng, Lei Wang, Hao Yang, Chong Peng, Peng Yan, Fumin Shen, Jingkuan Song, Heng Tao Shen, Xing Xu

Low-Resource NLP

Challenge:	Existing methods that adapt LVLMs to egocentric tasks overlook critical agent-environment interactions, limiting their ability to perform egoic reasoning.
Approach:	They propose a zero-shot paradigm to enhance egocentric reasoning by simulating human causal reasoning by formalizing ego-centric reasoning using a structural causal model.
Outcome:	The proposed method improves egocentric reasoning abilities on six tasks.

Finding the Dominant Winning Ticket in Pre-Trained Language Models (2022.findings-acl)

Copied to clipboard

Zhuocheng Gong, Di He, Yelong Shen, Tie-Yan Liu, Weizhu Chen, Dongyan Zhao, Ji-Rong Wen, Rui Yan

DeltaNet: Conditional Medical Report Generation for COVID-19 Diagnosis (2022.coling-1)

Challenge:	Existing studies on pre-trained language models show that they can fine-tune parameters but achieve good downstream performance.
Approach:	They find that a dominant winning ticket takes up 0.05% of the parameters and is transferable across different tasks.
Outcome:	The proposed model can achieve comparable performance with the full-parameter model, the authors show . the dominant winning ticket takes up 0.05% of the parameters, and the model is transferable across tasks, they show - the authors conclude .

Copied to clipboard

Xian Wu, Shuxin Yang, Zhaopeng Qiu, Shen Ge, Yangtian Yan, Xingwang Wu, Yefeng Zheng, S. Kevin Zhou, Li Xiao

Qsnail: A Questionnaire Dataset for Sequential Question Generation (2024.lrec-main)

Challenge:	X-ray and CT are the gold standard for COVID-19 diagnosis and treatment . however, due to the excessive number of patients, writing reports becomes a heavy burden for radiologists.
Approach:	They propose to use X-ray and CT to generate medical reports automatically . they evaluate DeltaNet on a COVID-19 dataset, where it outperforms state-of-the-art approaches .
Outcome:	The proposed system outperforms state-of-the-art methods on a COVID-19 dataset.

Copied to clipboard

Yan Lei, Liang Pang, Yuanzhuo Wang, Huawei Shen, Xueqi Cheng

Dialogue Summarization with Static-Dynamic Structure Fusion Graph (2023.acl-long)

Challenge:	Questionnaires are a professional research methodology used for qualitative and quantitative analysis of human opinions, preferences, and behaviors.
Approach:	They propose a questionnaire-based dataset that consists of 13,168 human-written questionnaires.
Outcome:	The proposed dataset contains 13,168 human-written questionnaires gathered from online platforms.

Copied to clipboard

Shen Gao, Xin Cheng, Mingzhe Li, Xiuying Chen, Jinpeng Li, Dongyan Zhao, Rui Yan

Beyond Prompting: An Efficient Embedding Framework for Open-Domain Question Answering (2025.acl-long)

Challenge:	Dialogue summarization is a challenging task since it has dynamic interaction nature and inconsistent information flow among various speakers.
Approach:	They propose a Static-Dynamic graph-based Dialogue Summarization model which fuses prior knowledge from human expertise and adaptively learns the graph structure in an end-to-end learning fashion.
Outcome:	The proposed model can help people capture the highlights of a semi-structured and multi-participant dialogue without reviewing the complex dialogue context.

Copied to clipboard

Zhanghao Hu, Hanqi Yan, Qinglin Zhu, Zhenyi Shen, Yulan He, Lin Gui

DRAGON: Domain-specific Robust Automatic Data Generation for RAG Optimization (2026.findings-eacl)

Challenge:	Large language models (LLMs) have recently pushed open-domain question answering (ODQA) to new heights.
Approach:	They propose an embedding-level framework that enhances both the retriever and the reader by reordering query representations via lightweight linear layers under an unsupervised contrastive learning objective.
Outcome:	The proposed framework outperforms baselines in accuracy and efficiency across three open-source LLMs, three retrieval methods, and four ODQA benchmarks.

Copied to clipboard

Haiyang Shen, Hang Yan, Zhongshi Xing, Mugeng Liu, Yue Li, Zhiyang Chen, Yuxiang Wang, Jiuzheng Wang, Yun Ma

Robustness in NLP

Challenge:	Existing retrieval-augmented generation paradigms rely heavily on public knowledge . Existing RAGs reliant on public information and often falter when faced with domain-specific queries.
Approach:	They propose a framework that combines a data-construction modeling approach with a scalable synthetic data-generation pipeline to optimize domain-specific retrieval performance.
Outcome:	The proposed framework optimizes domain-specific retrieval performance and bolsters retriever robustness.

How Large a Vocabulary Does Text Classification Need? A Variational Approach to Vocabulary Selection (N19-1)

Copied to clipboard

Wenhu Chen, Yu Su, Yilin Shen, Zhiyu Chen, Xifeng Yan, William Yang Wang

SQL Generation via Machine Reading Comprehension (2020.coling-main)

Challenge:	Using a pre-defined vocabulary is a common approach to selecting text inputs . however, using a large vocabulary is not economical, as it limits the model's applicability on computation-or memoryconstrained scenarios.
Approach:	They propose a more sophisticated variational vocabulary dropout to perform vocabulary selection . they propose two new metrics to measure area under accuracy-vocab curve and Vocab Size under X% accuracy drop .
Outcome:	The proposed framework outperforms the baselines on the vocabulary selection problem on multiple NLP classification tasks.

Copied to clipboard

Zeyu Yan, Jianqiang Ma, Yang Zhang, Jianping Shen

Programming Languages in NLP

Challenge:	Text-to-SQL systems can generate SQL queries given natural language questions.
Approach:	They propose a method that formulates a question answering problem as a query answering problem where different slots are predicted by a unified machine reading comprehension (MRC) model.
Outcome:	The proposed method can achieve competitive results on WikiSQL, suggesting it being a promising direction for text-to-SQl.

Reward Modeling Requires Automatic Adjustment Based on Data Quality (2024.findings-emnlp)

Copied to clipboard

Binghai Wang, Rui Zheng, Lu Chen, Zhiheng Xi, Wei Shen, Yuhao Zhou, Dong Yan, Tao Gui, Qi Zhang, Xuanjing Huang

Challenge:	Reinforcement Learning from Human Feedback (RLHF) is a method for aligning language models with human values.
Approach:	They propose a method that automatically adjusts reward modeling based on data quality . they use preference data to train a reward model that is more aligned with human values .
Outcome:	The proposed method stabilizes reward model training and significantly improves alignment performance on human preference datasets.

ModelScope-Agent: Building Your Customizable Agent System with Open-source Large Language Models (2023.emnlp-demo)

Copied to clipboard

Chenliang Li, He Chen, Ming Yan, Weizhou Shen, Haiyang Xu, Zhikai Wu, Zhicheng Zhang, Wenmeng Zhou, Yingda Chen, Chen Cheng, Hongzhu Shi, Ji Zhang, Fei Huang, Jingren Zhou

On the Step Length Confounding in LLM Reasoning Data Selection (2026.findings-acl)

Challenge:	Large language models (LLMs) have demonstrated remarkable capabilities to comprehend human intentions, engage in reasoning, and design planning-like behavior.
Approach:	They propose a framework that equips large language models with tool-use capabilities . they propose LLaMA and Chat-GLM as controllers, and a model-based agent framework .
Outcome:	The proposed framework equips open-source LLMs with tool-use capabilities . it provides a user-friendly system library with a customizable engine design .

Copied to clipboard

Bing Wang, Rui Miao, Chen Shen, Shaotian Yan, Kaiyuan Liu, Ximing Li, Xiaosong Yuan, Sinan Fan, Jun Zhang, Jieping Ye

MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware Multi-Segment Grounding (2026.acl-long)

Challenge:	Existing pipelines generate long reasoning data from more capable Large Language Models (LLMs) and apply manually heuristic or naturalness-based selection methods to filter high-quality samples.
Approach:	They propose to use supervised fine-tuning to generate long reasoning data from more capable Large Language Models and apply manually heuristic or naturalness-based selection methods to filter high-quality samples.
Outcome:	Experiments on four LLMs and five evaluation benchmarks show that the proposed approach is effective in mitigating step length confounding problem.

Copied to clipboard

Fuwen Luo, Shengfeng Lou, Chi Chen, Ziyue Wang, Chenliang Li, Weizhou Shen, Jiyue Guo, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Yang Liu

Visual Data in NLP

Challenge:	Existing methods for MLLMs struggle with fine-grained temporal reasoning . despite advances in video understanding, current methods struggle with time-sensitive tasks .
Approach:	They propose a time-stamp-aware multi-segment grounding method that enhances temporal understanding by introducing timestamps.
Outcome:	The proposed method outperforms existing methods on time-sensitive tasks and generalizes well across diverse temporal understanding scenarios.

CFBench: A Comprehensive Constraints-Following Benchmark for LLMs (2025.acl-long)

Copied to clipboard

Tao Zhang, ChengLIn Zhu, Yanjun Shen, Wenjing Luo, Yan Zhang, Hao Liang, Tao Zhang, Fan Yang, Mingan Lin, Yujing Qiao, Weipeng Chen, Bin Cui, Wentao Zhang, Zenan Zhou

Projecting Out the Malice: A Global Subspace Approach to LLM Detoxification (2026.acl-long)

Challenge:	Existing evaluations of Large Language Models (LLMs) focus on fragmented constraints or narrow scenarios, but they overlook the comprehensiveness and authenticity of constraints from the user’s perspective.
Approach:	They propose a Chinese Comprehensive Constraints Following Benchmark for LLMs that compiles constraints from real-world instructions and constructs a systematic framework for constraint types.
Outcome:	The proposed framework integrates multi-dimensional assessment criteria with requirement prioritization, covering various perspectives of constraints, instructions, and requirement fulfillment.

Copied to clipboard

Zenghao Duan, Zhiyi Yin, Zhichao Shi, Liang Pang, Shaoling Jing, Zihe Huang, Jiayi Wu, Yu Yan, Jingcheng Deng, Huawei Shen, Xueqi Cheng

Challenge:	Large language models (LLMs) exhibit exceptional performance but pose inherent risks of generating toxic content.
Approach:	They propose a method that removes toxic subspaces from FFN parameters . they propose to use a lightweight method to eliminate toxic subespaces .
Outcome:	The proposed method achieves SOTA detoxification while preserving general capabilities without large-scale retraining.

WISE: Weak-Supervision-Guided Step-by-Step Explanations for Multimodal LLMs in Image Classification (2025.emnlp-main)

Copied to clipboard

Yiwen Jiang, Deval Mehta, Siyuan Yan, Yaling Shen, Zimu Wang, Zongyuan Ge

FASTMATCH: Accelerating the Inference of BERT-based Text Matching (2020.coling-main)

Challenge:	Existing MCoT methods focus on inter-object reasoning, overlooking intra-object understanding crucial for image classification.
Approach:	They propose a Weak-supervision-guided Step-by-step Explanation method that reformulates MCoTs under weak supervision into concise, interpretable reasoning chains.
Outcome:	The proposed method improves interpretability by 37% and improves classification accuracy.

Copied to clipboard

Shuai Pang, Jianqiang Ma, Zeyu Yan, Yang Zhang, Jianping Shen

Detecting AI-Generated Content on Social Media with Multi-modal Language Models (2026.acl-industry)

Challenge:	Recent pre-trained language models have shown state-of-the-art accuracies in text matching.
Approach:	They propose a BERT-based text matching model where representations and interactions are decoupled . they propose generating final matching scores using a lightweight attention network .
Outcome:	Experiments show that the proposed model can achieve up to 100X speed-up to BERT and RoBERTa while keeping more up to 98.7% of the performance.

Copied to clipboard

Chenyang Yang, Shen Yan, Yibo Yang, Litao Hu, Yuchen Liu, Yuan Zeng, Hanchao Yu, Yinan Zhu, Sumedha Singla, Brian Vanover, Huijun Qian, Zihao Wang, Fujun Liu, Aashu Singh, Jianyu Wang, Xuewen Zhang

ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answering (2024.findings-emnlp)

Challenge:	Existing methods for AI-generated content detection face poor generalization to newer models, reliance on single modalities, and lack of interpretable explanations.
Approach:	They propose a model that curates diverse social media data and trains a vision-language model for detection and explanation.
Outcome:	The proposed model achieves state-of-the-art detection performance on public benchmarks and observes positive downstream impacts on user engagement.

Copied to clipboard

Yifan Wu, Lutao Yan, Leixian Shen, Yunhai Wang, Nan Tang, Yuyu Luo

VMSMO: Learning to Generate Multimodal Summary for Video-based News Articles (2020.emnlp-main)

Challenge:	Chart question answering (ChartQA) tasks are a critical part of visualization charts.
Approach:	They propose a chart question answering task that uses MLLMs to analyze charts . they propose 'Chain-of-Charts' textual prompt strategy that directs attention to visual elements .
Outcome:	The proposed model improves performance by 14.41% and 80% in low-level ChartQA tasks.

Copied to clipboard

Mingzhe Li, Xiuying Chen, Shen Gao, Zhangming Chan, Dongyan Zhao, Rui Yan

Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting (2024.findings-naacl)

Challenge:	Existing studies show that multimodal news can significantly improve users' sense of satisfaction for informativeness.
Approach:	They propose a task of Video-based Multimodal Summarization with Multimodal Output to solve this problem.
Outcome:	The proposed method can generate multimodal summaries with a single input . it can model the temporal dependency of video with semantic meaning of article .

Copied to clipboard

Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Le Yan, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang, Michael Bendersky

Enhancing Chain-of-Thought Reasoning with Critical Representation Fine-tuning (2025.acl-long)

Challenge:	Existing methods to rank documents using large language models do not understand these challenging ranking formulations.
Approach:	They propose to use Pairwise Ranking Prompting to improve ranking performance . they propose to outperform fine-tuned baseline rankers on benchmark datasets .
Outcome:	The proposed technique outperforms supervised baselines on benchmark datasets and outperformed other LLM-based solutions by over 10% on average.

Copied to clipboard

Chenxi Huang, Shaotian Yan, Liang Xie, Binbin Lin, Sinan Fan, Yue Xin, Deng Cai, Chen Shen, Jieping Ye

A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges (2025.findings-acl)

Challenge:	Representation Fine-tuning (ReFT) is a proposed method for improving parameter efficiency . however, it yields suboptimal performance, as fixed-position representations have uncertain impact on outputs .
Approach:	They propose a method that fine-tunes critical representations in a low-rank linear subspace while freezing the base model.
Outcome:	The proposed method improves accuracy of LLaMA-2-7B and ReFT by 18.2 and 3.8 on GSM8K.

Copied to clipboard

Yibo Yan, Jiamin Su, Jianxiang He, Fangteng Fu, Xu Zheng, Yuanhuiyi Lyu, Kun Wang, Shen Wang, Qingsong Wen, Xuming Hu

How to Write Summaries with Patterns? Learning towards Abstractive Summarization through Prototype Editing (D19-1)

Challenge:	This survey provides the first comprehensive analysis of mathematical reasoning in the era of multimodal large language models . integrating large language model with mathematical reasoning tasks is becoming significant as AI advances .
Approach:	They review over 200 studies published since 2021 and examine the state-of-the-art developments in Math-LLMs . they identify five major challenges hindering the realization of AGI in this domain .
Outcome:	The authors examine the state-of-the-art developments in Math-LLMs with a focus on multimodal settings.

Copied to clipboard

Shen Gao, Xiuying Chen, Piji Li, Zhangming Chan, Dongyan Zhao, Rui Yan

Text Generation

Challenge:	Extensive experiments on a large-scale real-world text summarization dataset show that PESG achieves the state-of-the-art performance in terms of both automatic metrics and human evaluations.
Approach:	They propose a model that learns summary patterns and prototype facts from a prototype document . they use a fact checker to estimate mutual information between the input document and generated summary .
Outcome:	Experiments on a large-scale real-world text summarization dataset show that PESG achieves state-of-the-art performance.

DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models (2025.emnlp-industry)

Copied to clipboard

Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, Zhaoxiang Liu, Shiguo Lian

Green & Sustainable NLP

Challenge:	Recent advances in slow-thinking reasoning models have shown exceptional performance in complex reasoning tasks.
Approach:	They propose a framework that enables models to automatically adjust Chain-of-Thought (CoT) length based on problem difficulty.
Outcome:	The proposed framework penalizes inefficiency on simple problems while incentivizing deep reasoning for complex ones.

Semantic matching for text classification with complex class descriptions (2023.emnlp-main)

Copied to clipboard

Brian De Silva, Kuan-Wen Huang, Gwang Lee, Karen Hovsepian, Yan Xu, Mingwei Shen

MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detection (2025.acl-industry)

Challenge:	Existing methods for text classification support zero-shot learning but not both . Existing approaches do not support zero or few-shot, and are insufficient for complex classes .
Approach:	They propose a method which rapidly adapts from seen classes to new/unseen ones . they use labels and complex class descriptions to perform zero- and few-shot learning .
Outcome:	The proposed method beats baselines on complex class descriptions by 22.48% . it also improves zero-shot learning by 4.29% .

Copied to clipboard

Yibo Yan, Shen Wang, Jiahao Huo, Philip S. Yu, Xuming Hu, Qingsong Wen

Discover and Prove: An Open-source Agentic Framework for Hard Mode Automated Theorem Proving in Lean 4 (2026.acl-long)

Challenge:	Multimodal Large Language Models (MLLMs) struggle with identifying and categorizing student errors in multimodal mathematical contexts.
Approach:	They propose a new framework that decomposes error detection into three phases with specialized agents.
Outcome:	The proposed framework shows higher accuracy in error step identification and 3% improvement in error categorization on real-world educational data.

Copied to clipboard

Chengwu Liu, Yichun Yin, Ye Yuan, Jiaxuan Xie, Botao Li, Siqi Li, Jianhao Shen, Yan Xu, Lifeng Shang, Ming Zhang

SportQA: A Benchmark for Sports Understanding in Large Language Models (2024.naacl-long)

Challenge:	Existing approaches to solving mathematical problems fall into two broad categories: informal methods and formal methods.
Approach:	They propose to use LLM natural-language reasoning to discover answers . they introduce Discover And Prove framework that rewrites Hard Mode statements into Easy Mode ones for existing ATP provers.
Outcome:	The proposed framework can be used to prove hard mode statements on ATP benchmarks.

Copied to clipboard

Haotian Xia, Zhengbang Yang, Yuqing Wang, Rhys Tracy, Yun Zhao, Dongdong Huang, Zezhi Chen, Yan Zhu, Yuan-fang Wang, Weining Shen

Position: LLMs Can be Good Tutors in English Education (2025.emnlp-main)

Challenge:	SportQA is a benchmark specifically designed for evaluating Large Language Models (LLMs) sports knowledge is characterized by its fast pace, variety of types, abundance of strategies, and rich player narratives .
Approach:	They propose a benchmark specifically designed for evaluating Large Language Models in the context of sports understanding.
Outcome:	The proposed benchmark aims to bridge the gap between existing and specialized benchmarks in sports understanding.

Copied to clipboard

Jingheng Ye, Shen Wang, Deqing Zou, Yibo Yan, Kun Wang, Hai-Tao Zheng, Ruitong Liu, Zenglin Xu, Irwin King, Philip S. Yu, Qingsong Wen

Deeper Insights Without Updates: The Power of In-Context Learning Over Fine-Tuning (2024.findings-emnlp)

Challenge:	Recent efforts to integrate large language models into English education lack adaptability to language learning.
Approach:	They argue that large language models can be effective tutors in English education . they encourage interdisciplinary research to explore these roles, fostering innovation and risks .
Outcome:	The proposed models can play three critical roles: 1) as data enhancers, 2) as task predictors, 3) as agents, enabling personalized and inclusive education.

Copied to clipboard

Qingyu Yin, Xuzheng He, Chak Tou Leong, Fan Wang, Yanzhao Yan, Xiaoyu Shen, Qiang Zhang

Shallow Focus, Deep Fixes: Enhancing Shallow Layers Vision Attention Sinks to Alleviate Hallucination in LVLMs (2025.emnlp-main)

Challenge:	Fine-tuning and in-context learning are two prevalent methods in imbuing large language models with task-specific knowledge.
Approach:	They propose to use a circuit shift theory to explain why in-context learning is superior to fine-tuning for tasks with implicit patterns.
Outcome:	The proposed method can grasp deep patterns and significantly improve accuracy on implicit patterns, compared with fine-tuning and in-context learning.

Copied to clipboard

Xiaofeng Zhang, Yihao Quan, Chen Shen, Chaochen Gu, Xiaosong Yuan, Shaotian Yan, Jiawei Cao, Hao Cheng, Kaijie Wu, Jieping Ye

Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learning (2024.acl-long)

Challenge:	Multimodal large language models (MLLMs) demonstrate excellent abilities for understanding visual information, but the hallucination remains a challenging problem.
Approach:	They propose a training-free approach to enhance vision attention sinks to facilitate convergence of the image token attention sink within shallow layers.
Outcome:	The proposed approach improves the convergence of the image token attention sink within shallow layers and strengthens the layer’s focus on the image itself.

Copied to clipboard

Yue Yu, Jiaming Shen, Tianqi Liu, Zhen Qin, Jing Nathan Yan, Jialu Liu, Chao Zhang, Michael Bendersky

SSP: Self-Supervised Post-training for Conversational Search (2023.findings-acl)

Challenge:	Recent advances in natural language processing (NLP) have witnessed the remarkable capabilities of Large Language Models (LLMs).
Approach:	They propose an Explanation-Aware Soft Ensemble framework to empower in-context learning with Large language models.
Outcome:	The proposed framework can be used to enhance in-context learning on seven natural language understanding tasks and four varying-size LLMs.

Copied to clipboard

Quan Tu, Shen Gao, Xiaolong Wu, Zhao Cao, Ji-Rong Wen, Rui Yan

Concise and Organized Perception Facilitates Reasoning in Large Language Models (2025.findings-naacl)

Challenge:	Existing methods to search for information are limited by data scarcity and lack contextual semantic understanding.
Approach:	They propose a new post-training paradigm with three self-supervised tasks to initialize the conversational search model to enhance dialogue structure and contextual semantic understanding.
Outcome:	The proposed method can initialize the conversational search model to enhance dialogue structure and contextual semantic understanding.

Copied to clipboard

Junjie Liu, Shaotian Yan, Chen Shen, Zhengdong Xiao, Liang Xie, Wenxiao Wang, Jieping Ye