Papers by Sheng Chen

Meta-learning via Language Model In-context Tuning (2022.acl-long)

Challenge:	End-to-end speech-to speech (S2S) dialogue systems face key challenges in incorporating external knowledge into their models.
Approach:	They propose a framework that directly retrieves relevant textual knowledge from speech queries.
Outcome:	The proposed framework improves the performance of end-to-end speech-tospeech dialogue systems while achieving higher retrieval efficiency.

Copied to clipboard

Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis, He He

Autoregressive Speech Synthesis without Vector Quantization (2025.acl-long)

Challenge:	Recent advances in large language models have reduced "task learning and prediction" to a simple sequence prediction problem.
Approach:	They propose a meta-learning method that recasts task adaptation and prediction as a sequence prediction problem.
Outcome:	The proposed method outperforms MAML on two classification tasks and improves on binaryClfs.

Copied to clipboard

Lingwei Meng, Long Zhou, Shujie Liu, Sanyuan Chen, Bing Han, Shujie Hu, Yanqing Liu, Jinyu Li, Sheng Zhao, Xixin Wu, Helen M. Meng, Furu Wei

Automatic, Meta and Human Evaluation for Multimodal Summarization with Multimodal Output (2024.naacl-long)

Challenge:	MELLE is a novel language modeling approach for text-to-speech synthesis that generates continuous tokens from text . authors demonstrate that it reduces the need for vector quantization and improves model robustness .
Approach:	They propose to autoregressively generate continuous mel-spectrogram frames directly from text condition, bypassing vector quantization.
Outcome:	The proposed model achieves superior performance across multiple metrics and is more streamlined.

Copied to clipboard

Haojie Zhuang, Wei Emma Zhang, Leon Xie, Weitong Chen, Jian Yang, Quan Sheng

Summarization

Challenge:	Multimodal summarization with multimodal output (MSMO) has attracted increasing research interest . evaluation is an emerging yet underexplored research topic .
Approach:	They propose a framework that studies three research questions of MSMO evaluation . they propose an automatic evaluation metric and a meta-evaluation benchmark dataset .
Outcome:	The proposed evaluation metric and human-annotated meta-evaluation benchmark are used to assess the quality of evaluation metrics and show the framework is effective.

Depression Detection on Social Media with Large Language Models (2025.emnlp-industry)

Copied to clipboard

Xiaochong Lan, Zhiguang Han, Yiming Cheng, Li Sheng, Jie Feng, Chen Gao, Yong Li

Think Smart, Not Hard: Difficulty Adaptive Reasoning for Large Audio Language Models (2026.findings-acl)

Challenge:	Existing methods for analyzing social media data lack a systematic integration of medical knowledge, causing a critical treatment gap.
Approach:	They propose a framework that leverages Large Language Models to integrate medical knowledge into social media data.
Outcome:	The proposed framework can be used to distinguish depression from transient mood changes.

Copied to clipboard

Zhichao Sheng, Shilin Zhou, Chen Gong, Zhenghua Li

Adaptive Attentional Network for Few-Shot Knowledge Graph Completion (2020.emnlp-main)

Challenge:	Existing methods to determine whether to perform reasoning lack fine-grained mechanisms to adapt reasoning length to problem complexity.
Approach:	They propose a difficulty-adaptive reasoning method that dynamically links reasoning length to the model’s perceived problem difficulty.
Outcome:	The proposed method reduces average reasoning length by 50%, achieving higher efficiency without sacrificing accuracy.

Copied to clipboard

Jiawei Sheng, Shu Guo, Zhenyu Chen, Juwei Yue, Lihong Wang, Tingwen Liu, Hongbo Xu

How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models (2025.findings-naacl)

Challenge:	Recent attempts to learn static representations of entities and references ignore their dynamic properties.
Approach:	They propose to learn static representations of entities and references ignoring their dynamic properties . a neighbor encoder learns entities' roles while a query-aware aggregator learns references' contributions .
Outcome:	The proposed approach achieves state-of-the-art results with different few-shot sizes.

Copied to clipboard

Jiyue Jiang, Pengan Chen, Liheng Chen, Sheng Wang, Qinghang Bao, Lingpeng Kong, Yu Li, Chuan Wu

OptiCo: Adaptive Distributed Training Optimization via Collaborative Agent Reasoning (2026.acl-long)

Challenge:	Cantonese has scant representation in NLP research, especially compared to other languages from similarly developed regions.
Approach:	They propose to evaluate Cantonese LLM performance in factual generation, mathematical logic, complex reasoning, and general knowledge in Cantonesian.
Outcome:	The proposed models will evaluate Cantonese's performance in factual generation, mathematical logic, complex reasoning, and general knowledge in Cantone.

Copied to clipboard

Sheng Chen, Tang Zhe, Weixing Zhang, Fei Yang, Yuanyuan. Wang, Tianlin Li, Yang Liu

A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery (2024.emnlp-main)

Challenge:	Existing distributed training frameworks are plagued by over-reliance on prior profiling and poor generalization across models/hardware.
Approach:	They propose a model-driven multi-agent framework that leverages Large Language Models to enable automatic and explainable distributed training strategy configuration.
Outcome:	The proposed framework outperforms expert-designed training strategies within 20 iterations.

Copied to clipboard

Yu Zhang, Xiusi Chen, Bowen Jin, Sheng Wang, Shuiwang Ji, Wei Wang, Jiawei Han

A Systematic Survey of Automatic Prompt Optimization Techniques (2025.emnlp-main)

Challenge:	Existing surveys on scientific LLMs focus on one or two fields or a single modality.
Approach:	They survey 260 scientific LLMs and examine their architectures and pre-training techniques . they also discuss commonalities and differences between LLM architectures .
Outcome:	The proposed model architectures and evaluation techniques are used to improve scientific discovery.

Copied to clipboard

Kiran Ramnath, Kang Zhou, Sheng Guan, Soumya Smruti Mishra, Xuan Qi, Zhengyuan Shen, Shuai Wang, Sangmin Woo, Sullam Jeoung, Yawei Wang, Haozhu Wang, Han Ding, Yuzhe Lu, Zhichao Xu, Yun Zhou, Balasubramaniam Srinivasan, Qiaojing Yan, Yueyan Chen, Haibo Ding, Panpan Xu, Lin Lee Cheong

CortexDebate: Debating Sparsely and Equally for Multi-Agent Debate (2025.findings-acl)

Challenge:	Recent advances in prompt engineering have created impediments for end users to adopt . however, prompt engineering remains an impedance due to rapid advances in models, tasks, and associated best practices.
Approach:	They propose to define APO as a 5-part unifying framework and categorize all relevant works based on their salient features.
Outcome:	The proposed framework aims to improve the performance of large language models on various tasks.

Copied to clipboard

Yiliu Sun, Zicheng Zhao, Sheng Wan, Chen Gong

Structured Data in NLP

Challenge:	Existing methods to improve the reasoning performance of LLMs suffer from two major shortcomings: too lengthy input contexts and overconfidence dilemma.
Approach:	They propose a method to debating among LLM agents using a sparse debator graph . they use a module called McKinsey-based Debate Matter to optimize the debators .
Outcome:	The proposed method has been well demonstrated across eight datasets from four task types.

KCAT: A Knowledge-Constraint Typing Annotation Tool (P19-3)

Copied to clipboard

Sheng Lin, Luye Zheng, Bo Chen, Siliang Tang, Zhigang Chen, Guoping Hu, Yueting Zhuang, Fei Wu, Xiang Ren

Named Entity Recognition

Challenge:	Recent years Natural Language Processing community has seen a surge of interest in fine-grained entity typing (FET) given an entity mention (i.e. a sequence of token spans representing an entity), FET aims at uncovering its contextdependent type.
Approach:	They propose an efficient Knowledge Constraint Fine-grained Entity Typing Annotation Tool which further improves the entity typing process through entity linking together with some practical functions.
Outcome:	The proposed tool improves the entity typing process by linking the candidate types with some practical functions.

Measuring Pointwise 𝒱-Usable Information In-Context-ly (2023.findings-emnlp)

Copied to clipboard

Sheng Lu, Shan Chen, Yingya Li, Danielle Bitterman, Guergana Savova, Iryna Gurevych

Challenge:	In-context learning (ICL) is a new learning paradigm that has gained popularity along with the development of large language models.
Approach:	They propose to adapt a recently proposed hardness metric, pointwise V-usable information (PVI), to an in-context version.
Outcome:	The proposed hardness metric is compared with the original model and is more efficient because it requires only a few exemplars and does not require fine-tuning.

From Introspection to Best Practices: Principled Analysis of Demonstrations in Multimodal In-Context Learning (2025.naacl-long)

Copied to clipboard

Nan Xu, Fei Wang, Sheng Zhang, Hoifung Poon, Muhao Chen

Low-Resource NLP

Challenge:	Motivated by in-context learning capabilities of Large Language Models (LLMs), multimodal LLMs with additional visual modality are also exhibited with similar ICL abilities when multiple image-text pairs are provided as demonstrations.
Approach:	They conduct systematic and principled evaluation of multimodal ICL for models of different scales on a broad spectrum of new yet critical tasks.
Outcome:	The proposed model performance improves on a broad spectrum of new yet critical tasks.

MaskTab: Scalable Masked Tabular Pretraining with Scaling Laws and Distillation for Industrial Classification (2026.findings-acl)

Copied to clipboard

Bo Zheng, Yudong Chen, Zihua Xiong, Shuai Fang, Peidong He, Yang Yang, Sheng Guo

Derailer-Rerailer: Adaptive Verification for Efficient and Reliable Language Model Reasoning (2025.findings-acl)

Challenge:	Tabular data is high-dimensional, riddled with missing entries, and rarely labeled at scale.
Approach:	They propose a unified pre-training framework for industrial-scale tabular data . MaskTab encodes missing values via dedicated learnable tokens .
Outcome:	The proposed framework outperforms XGBoost and MaskTab-L on industrial-scale . it achieves +5.04% AUC and +8.28% KS over prior art under rigorous scaling .

Copied to clipboard

Guangya Wan, Yuqi Wu, Hao Wang, Shengming Zhao, Jie Chen, Sheng Li

Developing and Utilizing a Large-Scale Cantonese Dataset for Multi-Tasking in Large Language Models (2025.findings-emnlp)

Challenge:	Existing prompting methods struggle with complex tasks and reasoning stability, limiting their practical deployment.
Approach:	They propose a framework that adaptively balances reasoning accuracy and computational efficiency by employing a lightweight Derailer mechanism to assess reasoning stability and selectively triggers an advanced Rerailer verification process only when necessary.
Outcome:	The proposed framework achieves significant accuracy improvements (8-11%) while maintaining 2-3 times better efficiency than existing verification methods.

Copied to clipboard

Jiyue Jiang, Alfred Kar Yin Truong, Yanyu Chen, Qinghang Bao, Sheng Wang, Pengan Chen, Jiuming Wang, Lingpeng Kong, Yu Li, Chuan Wu

OctoTools: A Multi-Agent Framework with Extensible Tools for Complex Reasoning (2026.acl-long)

Challenge:	Cantonese is considered a low-resource language due to the dominance of Mandarin . rich colloquial vocabulary of Cantone, English loanwords, and code-switching characteristics add to the complexity of corpus collection and processing.
Approach:	We collect Cantonese texts from open source corpora, Hong Kong-specific forums, Wikipedia . we refine the model through supervised fine-tuning on curated Cantonesian tasks .
Outcome:	The model achieves state-of-the-art (SOTA) performance on four Cantonese benchmarks.

Copied to clipboard

Pan Lu, Bowen Chen, Sheng Liu, Rahul Thapa, Joseph Boen, James Zou

MetaScale: Test-Time Scaling with Evolving Meta-Thoughts (2026.findings-acl)

Challenge:	Existing prompting methods for large language models (LLMs) are restricted to specialized domains, limited tool types, or require additional training data.
Approach:	They propose a training-free, user-friendly, and easily extensible multi-agent framework designed to tackle complex reasoning across diverse domains.
Outcome:	The proposed framework outperforms AutoGen, GPT-Functions, and LangChain by up to 10.6% when given the same set of tools.

Copied to clipboard

Qin Liu, Wenxuan Zhou, Nan Xu, James Y. Huang, Fei Wang, Sheng Zhang, Hoifung Poon, Muhao Chen

Unmasking Deceptive Visuals: Benchmarking Multimodal Large Language Models on Misleading Chart Question Answering (2025.emnlp-main)

Challenge:	Existing approaches impose fixed cognitive structures that enhance performance in specific tasks but lack adaptability across diverse scenarios.
Approach:	They propose a test-time scaling framework based on meta-thoughts to improve performance . meta-thinkts are adaptive thinking strategies tailored to a given task .
Outcome:	Experimental results show that MetaScale outperforms standard inference approaches . it can scale more effectively with increasing sampling budgets and produces more structured responses .

Copied to clipboard

Zixin Chen, Sicheng Song, KaShun Shum, Yanna Lin, Rui Sheng, Weiqi Wang, Huamin Qu

SceneLM: 3D-Aware Language Models for Editable 3D Scene Synthesis (2026.findings-acl)

Challenge:	Misleading visualizations can distort perception and lead to incorrect conclusions.
Approach:	They propose a large-scale multimodal dataset to evaluate MLLMs on misleading chart reasoning.
Outcome:	The proposed framework evaluates MLLMs on misleading chart reasoning on a large-scale multimodal dataset spanning 21 misleader types and 10 chart types . it contains 3,026 curated examples spanning standard chart code, CSV data, multiple-choice questions, and labeled explanations, validated through iterative MLML checks and exhausted expert human review.

Copied to clipboard

Xingbo Yao, Xiaoyu Chen, Doudou Zhang, Mingzhi Sheng, Boyuan Cao, Ying-Cong Chen, Hui Xiong

RECALL: REpresentation-aligned Catastrophic-forgetting ALLeviation via Hierarchical Model Merging (2025.emnlp-main)

Challenge:	Existing methods for synthesising 3D scenes from a single image are text-driven and lack precise metric understanding from images.
Approach:	They propose a language-model-based framework that grounds 3D scene synthesis in visual evidence by recovering an executable metric 3D layout directly from a single image.
Outcome:	The proposed framework recovers an executable metric 3D layout directly from an RGB image and instantiates, places, and edits objects for iterative refinement.

Copied to clipboard

Bowen Wang, Haiyuan Wan, Liwen Shi, Chen Yang, Peng He, Yue Ma, Haochen Han, Wenhao Li, Tiao Tan, Yongjian Li, Fangming Liu, Gong Yifan, Sheng Zhang

Agentic Episodic Control (2026.findings-acl)

Challenge:	Existing models that require task labels or performance trade-offs are susceptible to catastrophic forgetting.
Approach:	They propose a representation-aware model merging framework for continual learning without access to historical data.
Outcome:	The proposed framework outperforms baselines in knowledge retention and generalization across five NLP tasks and multiple continual learning scenarios.

Copied to clipboard

Xidong Yang, Wenhao Li, Junjie Sheng, Yun Hua, Haosheng Chen, Chuyun Shen, Xiangfeng Wang

Green & Sustainable NLP

Challenge:	Existing methods for reinforcement learning (RL) are limited by poor data efficiency and weak generalization.
Approach:	They propose a novel architecture that integrates large language models into episodic RL.
Outcome:	The proposed architecture achieves 2–6 higher data efficiency than baselines and is the only method to solve complex tasks like UnlockLocal with over 90% success.

IPR: Intelligent Prompt Routing with User-Controlled Quality-Cost Trade-offs (2025.emnlp-industry)

Copied to clipboard

Aosong Feng, Balasubramaniam Srinivasan, Yun Zhou, Zhichao Xu, Kang Zhou, Sheng Guan, Yueyan Chen, Xian Wu, Ninad Kulkarni, Yi Zhang, Zhengyuan Shen, Dmitriy Bespalov, Soumya Smruti Mishra, Yifei Teng, Darren Yow-Bang Wang, Haibo Ding, Lin Lee Cheong

Low-Resource NLP

Challenge:	Existing systems require users to manually select models or employ rigid routing rules that fail to capture the continuous spectrum of query complexity.
Approach:	They propose a quality-constrained intelligent prompt routing framework that automatically selects optimal models based on predicted response quality and user-specified tolerance levels.
Outcome:	The proposed framework achieves 43.9% cost reduction while maintaining quality parity with strongest model in the Claude family and processes requests with sub-150ms latency.

Towards Interpretable Tabular Reasoning: Enhancing LLM Reasoning on Tabular Data with Pre-Constructed Logic Graph (2026.acl-long)

Copied to clipboard

Lirong Gao, Zewei Yu, Zhongrui Yin, Qi Zhang, Yuke Zhu, Bo Zheng, Haobo Wang, Junbo Zhao, Gang Chen, Sheng Guo

Forget the Token and Pixel: Rethinking Gradient Ascent for Concept Unlearning in Multimodal Generative Models (2025.findings-acl)

Challenge:	Tabular data is used in fields such as finance and healthcare due to its heterogeneity and complexity.
Approach:	They propose a Logic-Graph-Enhanced LLM Reasoning framework that integrates the strengths of tree-based models and LLMs to improve their interpretability.
Outcome:	The proposed framework outperforms tree-based models and state-of-the-art LLMs on tabular prediction tasks, achieving superior accuracy and interpretability.

Copied to clipboard

Jiaqi Li, Chuanyi Zhang, Miaozeng Du, Hui Zhang, Yongrui Chen, Qianshan Wei, Junfeng Fang, Ruipeng Wang, Sheng Bi, Guilin Qi

Continual Contrastive Finetuning Improves Low-Resource Relation Extraction (2023.acl-long)

Challenge:	Gradient Ascent (GA) has emerged as a promising approach for concept unlearning in Multimodal Generative Models (MGMs).
Approach:	They propose a novel approach that selectively applies GA to targeted Conceptual Knowledge while preserving Natural Knowledge through Gradient Descent (GD).
Outcome:	The proposed approach removes Conceptual Knowledge and inadvertently diminishes Natural Knowledge, resulting in utility degradation.

Copied to clipboard

Wenxuan Zhou, Sheng Zhang, Tristan Naumann, Muhao Chen, Hoifung Poon

Semantic-Aware Logical Reasoning via a Semiotic Framework (2026.acl-long)

Challenge:	Relation extraction (RE) has been challenging in low-resource domains and with limited resources.
Approach:	They propose to pretrain and finetune the RE model using consistent objectives of contrastive learning.
Outcome:	The proposed method outperforms PLM-based RE classifier on two document-level RE datasets.

Copied to clipboard

Yunyao Zhang, Xinglang Zhang, Junxi Sheng, Wenbing Li, Junqing Yu, Yi-Ping Phoebe Chen, Wei Yang, Zikai Song

ObfusLM: Privacy-preserving Language Model Service against Embedding Inversion Attacks (2025.acl-long)

Challenge:	Existing studies largely overlook the interplay between logical complexity and semantic complexity, limiting their robustness under abstract propositions, ambiguous contexts, and conflicting stances.
Approach:	They propose a semiotic-square-guided framework that integrates automated deduction with reflective verification to manage logical complexity across deeper reasoning chains.
Outcome:	The proposed framework achieves state-of-the-art performance on RepublicQA with 6.25% average gain, and generalizes well to four mainstream logical reasoning benchmarks with an additional 7.05% improvement.

Copied to clipboard

Yu Lin, Ruining Yang, Yunlong Mao, Qizhi Zhang, Jue Hong, Quanwei Cai, Ye Wu, Huiqi Liu, Zhiyu Chen, Bing Duan, Sheng Zhong

Responsible & Trustworthy NLP

Challenge:	Recent studies show that obfuscation techniques for MLaaS are susceptible to embedding inversion attacks (EIAs).
Approach:	They propose a model obfuscation framework that protects client inputs from embedding inversion attacks by obliviously obbing models.
Outcome:	The proposed framework outperforms existing works in utility by 10% with a nearly 80% resistance rate against embedding inversion attacks.

LoRA Meets Dropout under a Unified Framework (2024.findings-acl)

Copied to clipboard

Sheng Wang, Liheng Chen, Jiyue Jiang, Boyang Xue, Lingpeng Kong, Chuan Wu

Green & Sustainable NLP

Challenge:	Parameter-efficientfinetuning (PEFT) has gained popularity as a lightweight approach for model customization.
Approach:	They propose a parameter-efficient dropout method that is overfitting-prone and parameter-freezed.
Outcome:	The proposed method is superior to existing methods and compares with transformer-specific methods.

Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking (2025.acl-long)

Copied to clipboard

Yilong Chen, Junyuan Shang, Zhenyu Zhang, Yanxi Xie, Jiawei Sheng, Tingwen Liu, Shuohuan Wang, Yu Sun, Hua Wu, Haifeng Wang

Can LLMs be Good Graph Judge for Knowledge Graph Construction? (2025.emnlp-main)

Challenge:	Large language models face inherent performance bottlenecks under parameter constraints . challenging tokens induce abrupt gradient spikes across layers, exposing stress points .
Approach:	They propose an inner thinking transformer that reimagines layer computations as implicit thinking steps.
Outcome:	Empirical results show that ITT outperforms Transformer/Loop variants in 11 benchmarks.

Copied to clipboard

Haoyu Huang, Chong Chen, Zeang Sheng, Yang Li, Wentao Zhang

Structured Data in NLP

Challenge:	Existing methods for converting unstructured text into structured Knowledge Graphs (KGs) have limitations such as large amount of noise, inaccurate knowledge, and hallucination .
Approach:	They propose a GraphJudge framework to reduce noise in real-world documents . they propose Graphjudge to fine-tune a LLM as a graph judge to enhance quality .
Outcome:	The proposed framework eliminates noise in real-world documents and improves the quality of generated KGs.

SpeechIQ: Speech-Agentic Intelligence Quotient Across Cognitive Levels in Voice Understanding by Large Language Models (2025.acl-long)

Copied to clipboard

Zhen Wan, Chao-Han Huck Yang, Yahan Yu, Jinchuan Tian, Sheng Li, Ke Hu, Zhehuai Chen, Shinji Watanabe, Fei Cheng, Chenhui Chu, Sadao Kurohashi

RGL: A Simple yet Effective Relation Graph Augmented Prompt-based Tuning Approach for Few-Shot Learning (2022.findings-naacl)

Challenge:	SIQ quantifies voice understanding abilities and provides unified comparisons between cascaded methods and end-to-end models.
Approach:	They propose a human cognition-inspired evaluation pipeline for voice understanding large language models (LLM_Voice) that quantifies voice understanding abilities and provides unified comparisons between cascaded methods and end-to-end models.
Outcome:	The proposed framework quantifies voice understanding abilities and provides unified comparisons between cascaded methods and end-to-end models, identifies annotation errors in existing benchmarks, and detects hallucinations in LLM_Voice.

Copied to clipboard

Yaqing Wang, Xin Tian, Haoyi Xiong, Yueyang Li, Zeyu Chen, Sheng Guo, Dejing Dou

Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons (2024.emnlp-main)

Challenge:	Pre-trained language models (PLMs) are a good starting point for downstream applications, but it is difficult to generalize them to new tasks given a few labeled samples.
Approach:	They propose to use Relation Graph augmented learning to improve the performance of few-shot natural language understanding tasks by rewriting the input sequence into a cloze question with masks.
Outcome:	Extensive experiments show that Relation Graph augmented learning (RGL) improves performance of prompt-based tuning strategies.

Copied to clipboard

Yifei Wang, Yuheng Chen, Wanting Wen, Yu Sheng, Linjing Li, Daniel Zeng

mDPO: Conditional Preference Optimization for Multimodal Large Language Models (2024.emnlp-main)

Challenge:	Recent advances in Large Language Models have underscored their exceptional reasoning prowess with natural language understanding across a broad spectrum of tasks.
Approach:	They examine whether Large Language Models actively recall or retrieve their internal repositories of factual knowledge when faced with reasoning tasks.
Outcome:	The proposed model improves reasoning performance while suppressing it leads to notable degradation.

Copied to clipboard

Fei Wang, Wenxuan Zhou, James Y. Huang, Nan Xu, Sheng Zhang, Hoifung Poon, Muhao Chen

BBScoreV2: Learning Time-Evolution and Latent Alignment from Stochastic Representation (2025.emnlp-main)

Challenge:	Recent studies have attempted to apply DPO to multimodal scenarios but have found it challenging to achieve consistent improvement.
Approach:	They propose a multimodal DPO objective that prevents the over-prioritization of language-only preferences by also optimizing image preference.
Outcome:	The proposed method significantly improves performance on two multimodal LLMs of different sizes and three widely used benchmarks.

Copied to clipboard

Tianhao Zhang, Zhecheng Sheng, Zhexiao Lin, Chen Jiang, Dongyeop Kang

Representation Learning

Challenge:	Autoregressive generative models are gaining traction in language tasks such as text generation and machine translation.
Approach:	They propose a likelihood-based evaluation metric that fits transformer-based model embeddings into a stochastic process and propose it as a probability-based metric.
Outcome:	The proposed model embeddings induce a "clustered-to-temporal ordered" mapping of language model representations in high-dimensional space, and this structure enhances performance on tasks such as temporal consistency evaluation and AI-generated content detection.

Cross-Document Event Coreference Resolution on Discourse Structure (2023.emnlp-main)

Copied to clipboard

Xinyu Chen, Sheng Xu, Peifeng Li, Qiaoming Zhu

Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling (2025.naacl-long)

Challenge:	Experimental results show that our proposed model outperforms several baselines and achieves the competitive performance with the start-of-the-art baselines.
Approach:	They propose to use discourse rhetorical structure constructor to construct tree structures to represent documents and a multi-layer perceptron to capture similarities of event mention pairs.
Outcome:	The proposed model outperforms baselines and achieves competitive performance with the start-of-the-art baselines.

Copied to clipboard

Guangya Wan, Yuqi Wu, Jie Chen, Sheng Li

Responsible & Trustworthy NLP

Challenge:	Large Language Models (LLMs) generate reasoning paths before answers, but lack a systematic approach to determine optimal number of samples or select the most faithful rationale.
Approach:	They propose a framework that evaluates the quality of reasoning and consistency of answers for each generated sample and uses criteria-based stopping and weighted majority voting to guide early stopping decisions and rationale selection.
Outcome:	The proposed framework outperforms existing methods while maintaining accuracy.

CoTD-PO: Chain-of-Thought Distillation with Preference Optimization (2025.findings-emnlp)

Copied to clipboard

Lujie Niu, Haochen Sun, Fangkun Zhao, Sheng Chen, Zimeng Bai, Jiawei Zhang, Caixia Yuan, Xiaojie Wang

MMAD:Multi-modal Movie Audio Description (2024.lrec-main)

Challenge:	Existing methods for chain-of-thought distillation suffer from a distribution mismatch between teacher-generated training trajectories and the student model's own generative distribution.
Approach:	They propose a framework that shifts the training paradigm from passive imitation to active trajectory exploration by allowing students to sample their own answer paths.
Outcome:	The proposed method outperforms standard CoT distillation baselines while mitigating mode collapse and preserving semantic diversity.

Copied to clipboard

Xiaojun Ye, Junhao Chen, Xiang Li, Haidong Xin, Chao Li, Sheng Zhou, Jiajun Bu

FinMRAGBench: A Realistic and Complex Benchmark for Multi-Modal RAG in Financial Document Analysis (2026.findings-acl)

Challenge:	Current methods of creating accessible movies rely on manual work, resulting in high costs and limited scalability.
Approach:	They propose a multi-modal movie audio description pipeline that generates narrations of information that is not accessible through unimodal hearing in movies.
Outcome:	The proposed pipeline surpasses existing baselines in performance on widely used datasets.

Copied to clipboard

Shouqing Yang, Qi Zhang, Yuhang Yang, Ruikang Xu, Yuwei Hou, Zhulin Jia, Lirong Gao, Haobo Wang, Jinglei Chen, Jiexiang Wang, Sheng Guo, Bo Zheng, Gang Chen

Information Retrieval

Challenge:	Existing benchmarks for realistic financial analysis fail to capture realistic financial situations involving cross-document retrieval, multi-page evidence integration, and diverse analytical tasks.
Approach:	They propose a multi-modal financial RAG benchmark that evaluates large language models in realistic financial analysis settings.
Outcome:	The proposed framework achieves the strongest overall performance across all models.

AMPO: Automatic Multi-Branched Prompt Optimization (2024.emnlp-main)

Copied to clipboard

Sheng Yang, Yurong Wu, Yan Gao, Zineng Zhou, Bin Zhu, Xiaodi Sun, Jian-Guang Lou, Zhiming Ding, Anbang Hu, Yuan Fang, Yunsong Li, Junyan Chen, Linjun Yang

Noisy-Labeled NER with Confidence Estimation (2021.naacl-main)

Challenge:	Existing prompt engineering techniques are limited to producing single flow instructions, struggling with handling diverse patterns.
Approach:	They propose an automatic prompt optimization method that iteratively develops a multi-branched prompt using failure cases as feedback.
Outcome:	The proposed method achieves the best results across five tasks and demonstrates significant optimization efficiency due to adoption of a minimal search strategy.

Copied to clipboard

Kun Liu, Yao Fu, Chuanqi Tan, Mosha Chen, Ningyu Zhang, Songfang Huang, Sheng Gao

SQL-Trail: Multi-Turn Reinforcement Learning with Interleaved Feedback for Text-to-SQL (2026.acl-long)

Challenge:	Recent studies in deep learning have shown significant progress in named entity recognition (NER) . however, most existing works assume clean data annotation, while real-world data typically involve a large amount of noises.
Approach:	They propose a confidence estimation approach for named entity recognition using noisy labels using local and global independence assumptions.
Outcome:	The proposed method marginalizes out labels of low confidence with a CRF model and integrates it into a self-training framework for boosting performance.

Copied to clipboard

Harper Hua, Zhen Han, Zhengyuan Shen, Meng-Chieh Lee, Sheng Guan, Qi Zhu, Sullam Jeoung, Yueyan Chen, Yunfei Bai, Shuai Wang, Vassilis N. Ioannidis, Huzefa Rangwala

Programming Languages in NLP

Challenge:	Recent large language models (LLMs) have significantly improved Text-to-SQL generation, but a gap remains between AI systems and human experts on challenging benchmarks such as BIRD-Sql.
Approach:	They propose a multi-turn reinforcement learning agentic framework for Text-to-SQL that uses execution feedback to iteratively refine its predictions.
Outcome:	The proposed framework outperforms proprietary systems on 7B and 14B models by **5% on average, underscoring the effectiveness of interactive, agentic workflows for robust Text-to-SQL generation.

Exploiting Entity BIO Tag Embeddings and Multi-task Learning for Relation Extraction with Imbalanced Data (P19-1)

Copied to clipboard

Wei Ye, Bo Li, Rui Xie, Zhonghao Sheng, Long Chen, Shikun Zhang

MR-ALIGN: Meta-Reasoning Informed Factuality Alignment for Large Reasoning Models (2026.findings-acl)

Challenge:	Existing methods to perform relation extraction are feature-based or kernel-based, but the results of our study show that they can improve the performance of a baseline model with more than 10% absolute increase in F1-score.
Approach:	They propose a multi-task architecture which jointly trains a model to perform relation identification with cross-entropy loss and relation classification with ranking loss.
Outcome:	The proposed model outperforms the state-of-the-art models on ACE 2005 Chinese and English corpus and significantly improves the performance of a baseline model with more than 10% increase in F1-score.

Copied to clipboard

Xinming Wang, Jian Xu, Bin Yu, Sheng Lian, yi Chen, Boran Wang, Yingjian Zhu, Hongzhu Yi, Hong-Ming Yang, Han Hu, Cheng-Lin Liu, Xu-Yao Zhang

Can LLMs Evaluate Complex Attribution in QA? Automatic Benchmarking using Knowledge Graphs (2025.acl-long)

Challenge:	Large reasoning models (LRMs) show strong capabilities in complex reasoning, yet their marginal gains on evidence-dependent factual questions are limited.
Approach:	They propose a Meta-Reasoning informed alignment framework that quantifies state-transition probabilities along the model’s thinking process and constructs a transition-aware implicit reward that reinforces beneficial reasoning patterns while suppressing defective ones at the atomic thinking segments.
Outcome:	Empirical evaluations of four factual QA datasets and one long-form factuality benchmark show that MR-ALIGN consistently improves accuracy and truthfulness while reducing misleading reasoning.

Copied to clipboard

Nan Hu, Jiaoyan Chen, Yike Wu, Guilin Qi, Hongru Wang, Sheng Bi, Yongrui Chen, Tongtong Wu, Jeff Z. Pan

Towards Scalable Lightweight GUI Agents via Multi-role Orchestration (2026.findings-acl)

Challenge:	Attributed Question Answering (AQA) has attracted wide attention, but there are several limitations in evaluating the attributions.
Approach:	They propose a large-scale benchmark containing comprehensive attribution categories . they compare 25 automatic evaluators with human evaluers and tested LLM evalators .
Outcome:	The proposed method can compare attributions with subtle differences and provide feedback to improve them.

Copied to clipboard

Ziwei Wang, Junjie Zheng, Leyang Yang, Sheng Zhou, Xiaoxuan Tang, Fang Zhouhua, Zhiwei Liu, Dajun Chen, Yong Li, Jiajun Bu

Challenge:	Advanced GUI agents suffer from prohibitive deployment costs on resource-constrained devices.
Approach:	They propose a lightweight GUI agent with GUI-specific knowledge and task scalability . LAMO-3B supports monolithic execution and MAS-style orchestration .
Outcome:	The proposed GUI agent LAMO-3B supports monolithic execution and MAS-style orchestration.

Learning Dynamic Context Augmentation for Global Entity Linking (D19-1)

Copied to clipboard

Xiyuan Yang, Xiaotao Gu, Sheng Lin, Siliang Tang, Yueting Zhuang, Fei Wu, Zhigang Chen, Guoping Hu, Xiang Ren

Knowledge Representation

Challenge:	Existing collective entity linking methods are expensive and often lack local context information.
Approach:	They propose a dynamic context-augmented inference model that can be used to make collective inference.
Outcome:	The proposed model can cope with different local EL models with different learning settings, base models, decision orders and attention mechanisms.

LeTS: Learning to Think-and-Search via Process-and-Outcome Reward Hybridization (2025.emnlp-main)

Copied to clipboard

Qi Zhang, Shouqing Yang, Lirong Gao, Hao Chen, Xiaomeng Hu, Jinglei Chen, Jiexiang Wang, Sheng Guo, Bo Zheng, Haobo Wang, Junbo Zhao

Calibrating Pseudo-Labeling with Class Distribution for Semi-supervised Text Classification (2025.emnlp-main)

Challenge:	Recent research focuses on integrating reasoning capabilities into the realm of retrieval-augmented generation (RAG) via outcome-supervised reinforcement learning (RL).
Approach:	They propose a process-level reward module to mitigate the unawareness of intermediate reasoning steps in outcome-level supervision without additional annotation.
Outcome:	The proposed framework can boost LLMs’ reasoning ability by integrating external knowledge sources through retrieval-augmented generation (RAG) The proposed model can mitigate the unawareness of intermediate reasoning steps in outcome-level supervision without additional annotation.

Copied to clipboard

Weiyi Yang, Richong Zhang, Junfan Chen, Jiawei Sheng

CoCGAN: Contrastive Learning for Adversarial Category Text Generation (2022.coling-1)

Challenge:	Existing studies develop effective pseudo-labeling methods, but they struggle with unlabeled data that have imbalanced classes mismatched with the labeled data.
Approach:	They propose to use pseudo-labeling to train text classification models with few labeled data and massive unlabeled data.
Outcome:	Empirical results show that the proposed model outperforms state-of-the-art methods on 3 common benchmarks.

Copied to clipboard

Xin Sheng, Linli Xu, Yinlong Xu, Changcun Bao, Huang Chen, Bo Ren

Context-faithful Prompting for Large Language Models (2023.findings-emnlp)

Challenge:	Experimental results on synthetic and real category text generation datasets demonstrate that CoCGAN can achieve significant improvements over the baseline category text generators.
Approach:	They propose to incorporate contrastive learning into adversarial category text generation by using a discriminator to optimize a contrastive learn objective to capture more flexible data-to-class relations and data- to-data relations among training samples.
Outcome:	The proposed model improves on synthetic and real category text generation datasets.

Copied to clipboard

Wenxuan Zhou, Sheng Zhang, Hoifung Poon, Muhao Chen

PRoLoRA: Partial Rotation Empowers More Parameter-Efficient LoRA (2024.acl-long)

Challenge:	Large language models encode parametric knowledge about world facts but overly rely on it can cause incorrect predictions in context-sensitive NLP tasks.
Approach:	They propose to use opinion-based prompts and counterfactual demonstrations to improve LLM faithfulness to contexts.
Outcome:	The proposed methods improve faithfulness to contexts using opinion-based prompts and counterfactual demonstrations.

Copied to clipboard

Sheng Wang, Boyang Xue, Jiacheng Ye, Jiyue Jiang, Liheng Chen, Lingpeng Kong, Chuan Wu

Green & Sustainable NLP

Challenge:	Partially Rotation-enhanced Low-Rank Adaptation (PRoLoRA) is an intra-layer sharing mechanism that circumvents the drawbacks of peer parameter-sharing methods.
Approach:	They propose a partially rotation-enhanced low-rank adaptation (PRoLoRA) that shares four components to reduce the cost of LoRA and improves model capacity.
Outcome:	Empirical results show that PRoLoRA outperforms LoRA on multiple instruction tuning datasets.

Improving Reasoning Capabilities in Small Models through Mixture-of-layers Distillation with Stepwise Attention on Key Information (2025.emnlp-main)

Copied to clipboard

Yao Chen, Jiawei Sheng, Wenyuan Zhang, Tingwen Liu