Papers by Yu Zeng

Improving End-to-End Speech Processing by Efficient Text Data Utilization with Latent Synthesis (2023.findings-emnlp)

Challenge:	Existing methods learned semantic representations with dual encoders or dual variational auto-encoders failed to capture the aligned semantics between question and answer.
Approach:	They propose to use two variational auto-encoders to generate questions with aligned answers and generating answers with align questions.
Outcome:	The proposed method outperforms the state-of-the-art answer retrieval method on SQuAD.

Copied to clipboard

Jianqiao Lu, Wenyong Huang, Nianzu Zheng, Xingshan Zeng, Yu Yeung, Xiao Chen

CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios (2025.emnlp-main)

Challenge:	Latent Synthesis is an efficient textual data utilization framework for end-to-end speech processing models . labeled speech data are scarcer and more expensive for collection compared to textual ones .
Approach:	They propose a textual data utilization framework for E2E speech processing models . they train a latent synthesizer to convert textual information into an intermediate latent representation .
Outcome:	The proposed framework improves on low-resource speech recognition and spoken language understanding tasks.

Copied to clipboard

Shiting Huang, Zhen Fang, Zehui Chen, Siyu Yuan, Junjie Ye, Yu Zeng, Lin Chen, Qi Mao, Feng Zhao

MentalSeek-Dx: Towards Progressive Hypothetico-Deductive Reasoning for Real-world Psychiatric Diagnosis (2026.acl-long)

Challenge:	a number of tools are used to perform complex tasks, but the tool utilization process can cause errors.
Approach:	They propose a critique evaluation benchmark for tool learning that analyzes function-calling errors on tool evaluation benchmarks.
Outcome:	The proposed critique evaluation benchmark holds diverse tool-use errors with varying complexities, which better reflects real-world scenarios.

Copied to clipboard

Xiao Sun, null Ymyang, Xinyi Jiang, Yu Tian, Junnan Zhu, Jiang Zhong, Qin Lei, Jingwang Huang, Haoyang Zeng, Xinyu Zhou, Xin Xiao, Kaiwen Wei

Multilingual Brain Surgeon: Large Language Models Can Be Compressed Leaving No Language behind (2024.lrec-main)

Challenge:	Mental health disorders represent a burgeoning global public health challenge . lack of ecological validity and fine-grained diagnostic supervision limits their utility .
Approach:	They propose a medical-specialized LLM trained to internalize clinical reasoning process through supervised trajectory construction and curriculum-based reinforcement learning.
Outcome:	The proposed model achieves state-of-the-art with only 14B parameters, establishing a clinically grounded framework for reliable psychiatric diagnosis.

Copied to clipboard

Hongchuan Zeng, Hongshen Xu, Lu Chen, Kai Yu

SafeSteer: A Decoding-level Defense Mechanism for Multimodal Large Language Models (2026.findings-acl)

Challenge:	Existing methods for MC focus on quantization and network pruning.
Approach:	They propose a calibration method that samples calibration data from various languages proportionally to the language distribution of the model training datasets.
Outcome:	The proposed method improves the performance of existing English-centric compression methods on the BLOOM multilingual LLM.

Copied to clipboard

Xinyi Zeng, Xue Yang, Jingyuan Zhang, Huanqian Yan, Xiang Chen, Kaiwen Wei, Hankun Kang, Yu Tian

Code-Switching Information Retrieval: Benchmarks, Analysis, and the Limits of Current Retrievers (2026.findings-acl)

Challenge:	Existing defense methods rely on fine-tuning or inefficient post-hoc interventions, limiting their ability to address novel attacks.
Approach:	They propose a decoding-level defense mechanism that employs a lightweight discriminator to iteratively steer the decoding process toward safety.
Outcome:	The proposed method improves safety performance by up to 33.40% without fine-tuning on multiple MLLMs.

Copied to clipboard

Qingcheng Zeng, Yuheng Lu, Zeqi Zhou, Heli Qi, Puxuan Yu, Fuheng Zhao, Hitomi Yanaka, Weihao Xuan, Naoto Yokoya

Multilinguality

Challenge:	a new study examines the performance of code-switching IR in monolingual contexts . code-witching is a pervasive linguistic phenomenon in global communication .
Approach:	They propose a benchmark to evaluate code-switching IR in monolingual contexts . they propose CS-MTEB, which measures performance declines of up to 27% .
Outcome:	The proposed benchmark shows that code-switching performance is degraded by 27% . the proposed benchmark is based on a dataset of mixed-language queries .

Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization (2023.acl-long)

Copied to clipboard

Pengcheng He, Baolin Peng, Song Wang, Yang Liu, Ruochen Xu, Hany Hassan, Yu Shi, Chenguang Zhu, Wayne Xiong, Michael Zeng, Jianfeng Gao, Xuedong Huang

Beyond Noise: Characterizing Creative Potential in Unverifiable LLM Hallucinations (2026.acl-long)

Challenge:	Z-Code++ is a pre-trained language model optimized for abstractive text summarization.
Approach:	They propose a pre-trained language model optimized for abstractive text summarization that uses a two-phase pre-training technique to improve model's performance.
Outcome:	The proposed model outperforms the competing models on low-resource summarization tasks in zero-shot and few-shot settings.

Copied to clipboard

Yu Yan, Chunhong Zhang, Haiyu Zhao, Ziyang Zeng, Zihao Liu, Yongkang Wu, Jianzhou Diao, YiJie Chen, Shujie Wang, Zheng Hu

PairCFR: Enhancing Model Training on Paired Counterfactually Augmented Data through Contrastive Learning (2024.acl-long)

Challenge:	Large Language Models generate outputs that extend beyond established knowledge . prior work does not characterize the unverifiable space as a whole .
Approach:	They propose a novelty-verifiability characterization that distinguishes Creative Synthesis from Groundless Fabrication by a conceptual creation task.
Outcome:	The proposed model distinguishes Creative Synthesis (Region A) from Groundless Fabrication (Regium B) it shows that Region A is non-negligible and robust, persisting across generation strategies, models, domains, and embedding choices.

Copied to clipboard

Xiaoqi Qiu, Yongjie Wang, Xu Guo, Zhiwei Zeng, Yu Yue, Yuhong Feng, Chunyan Miao

Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction (2023.emnlp-main)

Challenge:	Recent research shows that training with CAD may lead models to overly focus on modified features while ignoring other important contextual information.
Approach:	They propose to use contrastive learning to promote global feature alignment and learning counterfactual clues to improve model performance.
Outcome:	The proposed method outperforms the state-of-the-art on out-of distribution (OOD) datasets.

Copied to clipboard

Ji Qi, Chuchun Zhang, Xiaozhi Wang, Kaisheng Zeng, Jifan Yu, Jinxin Liu, Lei Hou, Juanzi Li, Xu Bin

i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data (2024.findings-naacl)

Challenge:	Existing evaluation benchmarks focus on pairwise matching, ignoring robustness . current models exhibit frustrating degradation, with a maximum drop of 23.43 F1 score .
Approach:	They propose a benchmark that simulates the evaluation of open information extraction models in the real world . they perform experiments on typical models published in the last decade and a representative large language model .
Outcome:	The proposed model is rated robust on a knowledge-invariant clique with different syntactic and expressive forms.

Copied to clipboard

Ziyi Yang, Mahmoud Khademi, Yichong Xu, Reid Pryzant, Yuwei Fang, Chenguang Zhu, Dongdong Chen, Yao Qian, Xuemei Gao, Yi-Ling Chen, Robert Gmyr, Naoyuki Kanda, Noel Codella, Bin Xiao, Yu Shi, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang

Automated Peer Reviewing in Paper SEA: Standardization, Evaluation, and Analysis (2024.findings-emnlp)

Challenge:	i-Code V2 is one of the first models capable of generating natural language from any combination of Vision, Language, and Speech data.
Approach:	They propose to create a model that can generate natural language from any combination of Vision, Language, and Speech data.
Outcome:	i-Code V2 matches or outperforms state-of-the-art single- and dual-modality baselines on 7 multimodal tasks.

Copied to clipboard

Jianxiang Yu, Zichen Ding, Jiaqi Tan, Kangyang Luo, Zhenmin Weng, Chenghua Gong, Long Zeng, RenJing Cui, Chengcheng Han, Qiushi Sun, Zhiyong Wu, Yunshi Lan, Xiang Li

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models (2024.acl-long)

Challenge:	Existing approaches to review scientific papers are limited by their content or quality . SEA is a framework for automated scientific review, but its contents are generic or partial.
Approach:	They propose a framework for automated scientific review using large language models . they propose to use a standardized review dataset to fine-tune an LLM to generate high-quality reviews.
Outcome:	The proposed framework can generate high-quality reviews from standardized datasets and improves on the existing feedback mechanisms.

Copied to clipboard

Damai Dai, Chengqi Deng, Chenggang Zhao, R.x. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y. Wu, Zhenda Xie, Y.k. Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, Wenfeng Liang

KBAlign: Efficient Self Adaptation on Specific Textual Knowledge Bases (2025.findings-emnlp)

Challenge:	Mixture-of-Experts (MoE) architectures face challenges in ensuring expert specialization . despite the promising performance, scaling language models to an extremely large scale is associated with exceedingly high computational costs.
Approach:	They propose an architecture that allows for ultimate expert specialization by segmenting experts into mN ones and activating mK from them.
Outcome:	The proposed architecture achieves comparable performance with GShard with 2B parameters and computation.

Copied to clipboard

Zheni Zeng, Yuxuan Chen, Shi Yu, Ruobing Wang, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun

Judge as A Judge: Improving the Evaluation of Retrieval-Augmented Generation through the Judge-Consistency of Large Language Models (2025.findings-acl)

Challenge:	Existing methods for retrieval-augmented generation (RAG) are limited and fine-tuning incurs prohibitive costs of external signals.
Approach:	They propose a self-supervised framework that enhances RAG systems through efficient model adaptation.
Outcome:	The proposed framework achieves 90% of the performance gain obtained through GPT-4-supervised adaptation while relying entirely on self-annotation of much smaller models.

Copied to clipboard

Shuliang Liu, Xinze Li, Zhenghao Liu, Yukun Yan, Cheng Yang, Zheni Zeng, Zhiyuan Liu, Maosong Sun, Ge Yu

Empowering GraphRAG with Knowledge Filtering and Integration (2025.emnlp-main)

Challenge:	Existing evaluation metrics cannot fairly evaluate the outputs of RAG models during training and evaluation.
Approach:	They propose a method which prompts LLMs to generate different judgments based on various combinations of judgment dimensions and utilizes the judge-consistency to evaluate these judgments.
Outcome:	The proposed method generates more accurate evaluations for RAG models across different RAG model and datasets.

Copied to clipboard

Kai Guo, Harry Shomer, Shenglai Zeng, Haoyu Han, Yu Wang, Jiliang Tang

The Pragmatic Mind of Machines: Tracing the Emergence of Pragmatic Competence in Large Language Models (2026.eacl-long)

Challenge:	Large language models suffer from knowledge gaps and hallucinations, resulting in incorrect or poor reasoning.
Approach:	They propose Graph retrieval-augmented generation (GraphRAG) which integrates structured knowledge from external graphs to enhance model's reasoning.
Outcome:	Experiments on knowledge graph QA tasks show that GraphRAG significantly improves reasoning performance across multiple backbone models.

Copied to clipboard

Kefan Yu, Qingcheng Zeng, Weihao Xuan, Wanxin Li, Jingyi Wu, Rob Voigt

Hetero-Designer: Automated Design of Multi-Agent Systems with Heterogeneous LLMs (2026.acl-long)

Challenge:	Current large language models (LLMs) have demonstrated emerging capabilities in social intelligence tasks, including implicature resolution and theory-of-mind reasoning.
Approach:	They introduce a dataset grounded in the pragmatic concept of alternatives to evaluate whether large language models can accurately infer nuanced speaker intentions.
Outcome:	The proposed model can infer nuanced speaker intentions by inferring the speaker’s intended meaning and explaining when and why a speaker would choose one utterance over its alternative.

Copied to clipboard

Zhiheng Zhang, Yuanzhe Zhang, Bohan Yu, Daojian Zeng, Kang Liu, Jun Zhao

Challenge:	Existing approaches to design LLM-based Multi-agent systems are constrained by homogeneous LLMs.
Approach:	They propose an automated design of heterogeneous-LLMs-based MAS with a binary-star transformer and an autoregressive graph generation pipeline.
Outcome:	The proposed pipeline is high-performing on various benchmarks and extensible to unseen LLMs and roles.

Corrupted but Not Broken: Understanding and Mitigating the Negative Impacts of Corrupted Data in Visual Instruction Tuning (2025.emnlp-main)

Copied to clipboard

Yunhao Gou, Hansi Yang, Zhili Liu, Kai Chen, Yihan Zeng, Lanqing Hong, Zhenguo Li, Qun Liu, Bo Han, James Kwok, Yu Zhang

HyperEdit: Unlocking Instruction-based Text Editing in LLMs via Hypernetworks (2026.findings-acl)

Challenge:	Visual Instruction Tuning (VIT) aims to enhance Multimodal Large Language Models (MLLMs), but its effectiveness is often compromised by corrupted datasets with issues such as hallucinated content and poor OCR quality.
Approach:	They propose a corruption-robust training paradigm that surpasses existing strategies for mitigating the effects of corrupted data.
Outcome:	The proposed training paradigm surpasses existing strategies for mitigating the effects of corrupted data.

Copied to clipboard

Yiming Zeng, Jinghan Cao, Zexin Li, Wanhao Yu, Zhankai Ye, Dawei Xiang, Ting Hua, Xin Liu, Shangqian Gao, Tingting Yu

Text Generation

Challenge:	Existing approaches treat instruction-based text editing as a generic text generation problem. Existing methods either over-edit or fail to apply modifications consistently.
Approach:	They propose a framework that processes each editing request to best align with it.
Outcome:	The proposed framework achieves 9% improvement over the state-of-the-art model.

Harnessing Consistency for Robust Test-Time LLM Ensemble (2026.findings-eacl)

Copied to clipboard

Zhichen Zeng, Qi Yu, Xiao Lin, Ruizhong Qiu, Xuying Ning, Tianxin Wei, Yuchen Yan, Jingrui He, Hanghang Tong

Robustness in NLP

Challenge:	Existing efforts to improve LLM ensemble quality have focused on model consistency, but failures are often due to heterogeneous tokenization schemes and varying model expertise.
Approach:	They propose a plug-and-play technique that harnesses model consistency for robust LLM ensemble.
Outcome:	The proposed technique improves ensemble performance and robustness against erroneous signals.

Marco-Bench-MIF: On Multilingual Instruction-Following Capability of Large Language (2025.acl-long)

Copied to clipboard

Bo Zeng, Chenyang Lyu, Sinuo Liu, Mingyan Zeng, Minghao Wu, Xuanfan Ni, Tianqi Shi, Yu Zhao, Yefeng Liu, Chenyu Zhu, Ruizhe Li, Jiahui Geng, Qing Li, Yu Tong, Longyue Wang, Weihua Luo, Kaifu Zhang

Multilinguality

Challenge:	Existing datasets for instruction-following are monolingual and centered on English . existing data are unable to capture linguistic and cultural subtle differences .
Approach:	They propose an extension of IFEval to a localized multilingual version called Marco-Bench-MIF . their benchmark addresses linguistic constraints and cultural references via translation and verification .
Outcome:	The proposed extension of IFEval to a localized multilingual version covers 30 languages with varying levels of localization.

RRAtention: Dynamic Block Sparse Attention via Per-Head Round-Robin Shifts for Long-Context Inference (2026.acl-long)

Copied to clipboard

Siran Liu, Guoxia Wang, Sa Wang, Jinle Zeng, Haoyang Xie, Siyu Lou, Jiabin Yang, Dianhai Yu, Haifeng Wang, Chao Yang

Visual Data in NLP

Challenge:	Existing approaches to dynamic sparse attention require preprocessing, lack global evaluation, violate query independence, or incur high computational overhead.
Approach:	They propose a dynamic sparse attention method that achieves all desirable properties through a head round-robin (RR) sampling strategy.
Outcome:	Experiments on natural language understanding and multimodal video comprehension show that the proposed method achieves 2.4 speedup at 128K context length outperforming existing methods.

CLUE: A Chinese Language Understanding Evaluation Benchmark (2020.coling-main)

Copied to clipboard

Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Kyle Richardson, Zhenzhong Lan

Challenge:	Existing language evaluation benchmarks for English are limited to English . lack of such benchmarks makes it difficult to replicate success in other languages .
Approach:	They introduce a large-scale Chinese language understanding evaluation benchmark . the benchmark uses a set of current state-of-the-art pre-trained Chinese models .
Outcome:	The first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark is released . the benchmark evaluates models across a wide range of tasks on original Chinese text . existing language evaluation benchmarks are mostly limited to English .

Faceted Hierarchy: A New Graph Type to Organize Scientific Concepts and a Construction Method (D19-53)

Copied to clipboard

Qingkai Zeng, Mengxia Yu, Wenhao Yu, JinJun Xiong, Yiyu Shi, Meng Jiang

Structured Data in NLP

Challenge:	faceted concept hierarchy is a structure of parent-child relationships . concepts are expected to be organized in a hierarchical structure for student learning .
Approach:	They propose a faceted concept hierarchy that aims to build facets from scientific literature.
Outcome:	The proposed hierarchy is more complete than "type-of" relations, and resolves conflicts by maintaining the acyclic structure of a hierarchy.

RESIN-11: Schema-guided Event Prediction for 11 Newsworthy Scenarios (2022.naacl-demo)

Copied to clipboard

Xinya Du, Zixuan Zhang, Sha Li, Pengfei Yu, Hongwei Wang, Tuan Lai, Xudong Lin, Ziqi Wang, Iris Liu, Ben Zhou, Haoyang Wen, Manling Li, Darryl Hannan, Jie Lei, Hyounghun Kim, Rotem Dror, Haoyu Wang, Michael Regan, Qi Zeng, Qing Lyu, Charles Yu, Carl Edwards, Xiaomeng Jin, Yizhu Jiao, Ghazaleh Kazeminejad, Zhenhailong Wang, Chris Callison-Burch, Mohit Bansal, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Martha Palmer, Heng Ji

Information Extraction & Text Mining

Challenge:	Existing methods for event prediction are incomplete and noisy.
Approach:	They propose to use news-related event schemas to extract newsworthy events . they build a demo website and include a video demonstrating the framework .
Outcome:	The proposed framework can be applied to a wide variety of newsworthy scenarios.

Exploring Concept Depth: How Large Language Models Acquire Knowledge and Concept at Different Layers? (2025.coling-main)

Copied to clipboard

Mingyu Jin, Qinkai Yu, Jingyuan Huang, Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, Fan Yang, Mengnan Du, Yongfeng Zhang

DPN-LE: Dual Personality Neuron Localization and Editing for Large Language Models (2026.findings-acl)

Challenge:	Large language models have shown remarkable performances across a wide range of tasks, but mechanisms by which they encode tasks of varying complexity remain poorly understood.
Approach:	They propose to explore the possibility that LLMs process concepts in different layers . they propose to categorize concepts based on their level of abstraction .
Outcome:	The proposed model can process complex concepts in shallow layers, the authors show . the proposed model could be used to prob complex tasks in shallow ones .

Copied to clipboard

Lifan Zheng, Xue Yang, Jiawei Chen, Chenyan WU, Jingyuan Zhang, Fanheng Kong, Xinyi Zeng, Xiang Chen, Yu Tian

Technical Question Answering across Tasks and Domains (2021.naacl-industry)

Challenge:	Current methods for editing personality traits in large language models can change personalities but reduce performance.
Approach:	They propose a novel paradigm for personality editing that locates and edits LLM neurons and enables competitive personality control at inference time.
Outcome:	Experiments on LLaMA-3-8B-Instruct and Qwen2.5-7B-instruct show that the proposed approach can improve performance and improve performance.

Copied to clipboard

Wenhao Yu, Lingfei Wu, Yu Deng, Qingkai Zeng, Ruchi Mahindru, Sinem Guven, Meng Jiang

ThinkNote: Enhancing Knowledge Integration and Utilization of Large Language Models via Constructivist Cognition Modeling (2026.findings-eacl)

Challenge:	Existing methods for technical QA have a limited data size and question and answer overlaps .
Approach:	They propose a framework of deep transfer learning to address technical QA across tasks and domains using document retrieval and reading comprehension tasks.
Outcome:	The proposed framework performs better than state-of-the-art methods on the TechQA task.

Copied to clipboard

Zhipeng Xu, Zhenghao Liu, Yukun Yan, Shuo Wang, Shi Yu, Zheni Zeng, Chaojun Xiao, Zhiyuan Liu, Ge Yu, Chenyan Xiong

Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons (2024.emnlp-main)

Challenge:	Large Language Models (LLMs) exhibit suboptimal behaviors and inconsistencies when exposed to unfamiliar external information, underscoring their limitations in effectively leveraging such knowledge.
Approach:	They propose a framework that enhances the external knowledge utilization of Large Language Models through a two-stage constructivist cognitive modeling process.
Outcome:	The proposed framework achieves a 10% improvement over baseline methods on various question-answering benchmarks.

Copied to clipboard

Yifei Wang, Yuheng Chen, Wanting Wen, Yu Sheng, Linjing Li, Daniel Zeng

Learning Confidence for Transformer-based Neural Machine Translation (2022.acl-long)

Challenge:	Recent advances in Large Language Models have underscored their exceptional reasoning prowess with natural language understanding across a broad spectrum of tasks.
Approach:	They examine whether Large Language Models actively recall or retrieve their internal repositories of factual knowledge when faced with reasoning tasks.
Outcome:	The proposed model improves reasoning performance while suppressing it leads to notable degradation.

Copied to clipboard

Yu Lu, Jiali Zeng, Jiajun Zhang, Shuangzhi Wu, Mu Li

Machine Translation

Challenge:	A well-calibrated confidence estimate is not sufficient for neural machine translation (NMT) where probabilities from softmax distribution fail to describe when the model is probably mistaken.
Approach:	They propose an unsupervised confidence estimate learning jointly with the training of a neural machine translation model to quantify confidence.
Outcome:	The proposed model outperforms standard label smoothing and can predict failures in two real-world scenarios.

Multi-View Incongruity Learning for Multimodal Sarcasm Detection (2025.coling-main)

Copied to clipboard

Diandian Guo, Cong Cao, Fangfang Yuan, Yanbing Liu, Guangjie Zeng, Xiaoyan Yu, Hao Peng, Philip S. Yu

Sentiment Analysis

Challenge:	Existing methods for multimodal sarcasm detection rely on spurious correlations, demonstrating poor generalizability beyond training environments.
Approach:	They propose a method that integrates multimodal incongruities via contrastive learning for multimodal sarcasm detection by using three views to drive multi-view learning.
Outcome:	The proposed method outperforms existing methods on benchmark datasets and shows that it is more generalizable than existing methods.

Type-enriched Hierarchical Contrastive Strategy for Fine-Grained Entity Typing (2022.coling-1)

Copied to clipboard

Xinyu Zuo, Haijin Liang, Ning Jing, Shuang Zeng, Zhou Fang, Yu Luo

Information Extraction & Text Mining

Challenge:	Experimental results show that fine-grained entity typing (FET) can be used to deduce specific semantic types of entities.
Approach:	They propose a type-enriched hierarchical contrastive strategy to model type differences . their method can make type information directly perceptible and improve distinguishability .
Outcome:	The proposed method can model the differences between hierarchical types and distinguish multi-grained similar types at different granularities.

LoRE: Enhancing Search Relevance with Progressive Chain-of-Thought and Preference Alignment (2026.findings-acl)

Copied to clipboard

Chenji Lu, Zhuo Chen, Hui Zhao, Zhiyuan Zeng, Gang Zhao, Junjie Ren, null Lihaoran, Songyan Liu, Pengjie Wang, Chuan Yu, Jian Xu, Bo Zheng

ARISE: An Adaptive Resolution-Aware Metric for Test-Time Scaling Evaluation in Large Reasoning Models (2026.findings-acl)

Challenge:	E-commerce search relevance is a critical component of retrieval systems.
Approach:	They propose a large-generative model for search relevance that trains reasoning knowledge, multi-modal understanding and rule awareness into three core competencies.
Outcome:	The proposed model outperforms GPT-5 in Macro-F1 and achieves 27% online gain.

Copied to clipboard

Zhangyue Yin, Qiushi Sun, Zhiyuan Zeng, Zhiyuan Yu, Qipeng Guo, Xuanjing Huang, Xipeng Qiu

Step-by-Step Mastery: Enhancing Soft Constraint Following Ability of Large Language Models (2025.findings-acl)

Challenge:	Existing evaluation methods for test-time scaling are limited.
Approach:	They propose an adaptive resolution-aware scaling evaluation metric specifically designed to assess the test-time scaling effectiveness of large reasoning models.
Outcome:	The proposed metric provides a reliable and fine-grained measurement of test-time scaling capabilities, revealing significant variations in scaling efficiency across models.

Copied to clipboard

Qingyu Ren, Jie Zeng, Qianyu He, Jiaqing Liang, Yanghua Xiao, Weikang Zhou, Zeye Sun, Fei Yu

Mixed-Lingual Pre-training for Cross-lingual Summarization (2020.aacl-main)

Challenge:	In real-world scenarios, user instructions often contain soft constraints, which are semantically related and cannot be rule-based verified, posing challenges for large language models.
Approach:	They propose a pipeline to construct datasets with high-quality outputs for instructions containing soft constraints automatically and use Direct Preference Optimization (DPO) as the training method.
Outcome:	The proposed model improves the LLMs' soft constraint following ability by using direct preference optimization (DPO) and constraint quantity.

Copied to clipboard

Ruochen Xu, Chenguang Zhu, Yu Shi, Michael Zeng, Xuedong Huang

Text Generation

Challenge:	Cross-lingual summarization (CLS) aims at producing a summary in the target language for an article in the source language.
Approach:	They propose a mixed-lingual pre-training scheme that leverages both cross-lingual tasks such as translation and monolingual tasks like masked language models.
Outcome:	The proposed model improves on the translation and masked language models with no task-specific components and saves memory.

RepoShapley: Shapley-Enhanced Context Filtering for Repository-Level Code Completion (2026.findings-acl)

Copied to clipboard

Yu Huo, Kun Zeng, Siyu Zhang, Yuquan LU, Cheng Yang, Yifu Guo, Xiaoying Tang

Programming Languages in NLP

Challenge:	Large language models have strong reasoning, coding, and generation capabilities, but retrieval-augmented generation remains difficult under fixed context budgets.
Approach:	They propose a coalition-aware context filtering framework supervised by Shapley-style marginal contributions that captures sign effects via teacher-forced probing and computes exact Shaply values for small retrieval sets.
Outcome:	Experiments show that RepoShapley improves completion quality while reducing harmful context and unnecessary retrieval.

ACEBench: A Comprehensive Evaluation of LLM Tool Usage (2025.findings-emnlp)

Copied to clipboard

Chen Chen, Xinlong Hao, Weiwen Liu, Xu Huang, Xingshan Zeng, Shuai Yu, Dexun Li, Yuefeng Huang, Xiangcheng Liu, Wang Xinzhi, Wu Liu

Dialogue Systems & Conversational Agents

Challenge:	Existing benchmarks for evaluating LLMs’ tool usage face several limitations: limited evaluation scenarios, lacking assessments in real multi-turn dialogue contexts; narrow evaluation dimensions, with insufficient detailed assessments of how LLM use tools; and reliance on LLM or real API executions for evaluation, which introduces significant overhead.
Approach:	ACEBench is a benchmark for evaluating tool usage in Large Language Models . it categorizes data into three primary types based on evaluation methodology: Normal, Special, and Agent.
Outcome:	ACEBench categorizes data into three primary types based on evaluation methodology: Normal, Special, and Agent.

Enhancing Large Vision-Language Models with Ultra-Detailed Image Caption Generation (2025.emnlp-main)

Copied to clipboard

Yu Zeng, Yukun Qi, Yiming Zhao, Xikun Bao, Lin Chen, Zehui Chen, Shiting Huang, Jie Zhao, Feng Zhao

Document Classification for COVID-19 Literature (2020.findings-emnlp)

Challenge:	Existing pipelines for generating high-quality, ultra-detailed image captions are limited by the scarcity of image caption data.
Approach:	They propose a pipeline for generating high-quality, ultra-detailed image captions that integrates both pre-processing and post-processor stages.
Outcome:	The proposed pipeline improves LVLMs' perception and cognitive abilities across multiple vision-language benchmarks.

Copied to clipboard

Bernal Jimenez Gutierrez, Jucheng Zeng, Dongdong Zhang, Ping Zhang, Yu Su

Bridging the Editing Gap in LLMs: FineEdit for Precise and Targeted Text Modifications (2025.findings-emnlp)

Challenge:	a global pandemic has made it more important than ever to quickly and accurately retrieve relevant scientific literature for effective consumption by researchers in a wide variety of fields.
Approach:	They analyze a LitCovid dataset to find out how classification models can help organize COVID-19 research papers.
Outcome:	The proposed model outperforms all baseline models on the LitCovid dataset . it also outperformed BioBERT and other models with micro-F1 and accuracy scores of 86% and 75% .

Copied to clipboard

Yiming Zeng, Wanhao Yu, Zexin Li, Tao Ren, Yu Ma, Jinghan Cao, Xiyan Chen, Tingting Yu

Challenge:	a recent study shows that large language models can perform precise text editing tasks.
Approach:	InstrEditBench is a benchmark dataset that compares 30,000 structured editing tasks . experimental evaluations show FineEdit outperforms state-of-the-art models .
Outcome:	The proposed model outperforms state-of-the-art models on single-turn edits and mistral-7B-OpenOrca on direct edits.

Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models (2023.findings-emnlp)

Copied to clipboard

Zhihan Zhang, Shuohang Wang, Wenhao Yu, Yichong Xu, Dan Iter, Qingkai Zeng, Yang Liu, Chenguang Zhu, Meng Jiang

RankCoT: Refining Knowledge for Retrieval-Augmented Generation through Ranking Chain-of-Thoughts (2025.acl-long)

Challenge:	Large language models can perform a wide range of tasks by following natural language instructions without task-specific fine-tuning.
Approach:	They propose a method to automatically improve the quality of LLM instructions . they leverage the generative ability of LMS to generate diverse candidate instructions based on a scoring model trained on 575 existing NLP tasks.
Outcome:	The proposed method surpasses human-written and LLM-generated instructions on 118 out-of-domain tasks.

Copied to clipboard

Mingyan Wu, Zhenghao Liu, Yukun Yan, Xinze Li, Shi Yu, Zheni Zeng, Yu Gu, Ge Yu

Text Generation

Challenge:	Retrieval-Augmented Generation (RAG) models enable Large Language Models to access external knowledge.
Approach:	They propose a knowledge refinement method that incorporates reranking signals to generate CoT-based summarization based on query and retrieval documents.
Outcome:	RankCoT generates CoT-based summarization based on query and all retrieval documents . Rank CoT incorporates a self-reflection mechanism that refines the outputs .

PromptARA: Improving Deep Representation in Hybrid Automatic Readability Assessment with Prompt and Orthogonal Projection (2023.findings-emnlp)

Copied to clipboard

Jinshan Zeng, Xianglong Yu, Xianchao Tong, Wenyan Xiao

MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation (2025.emnlp-main)

Challenge:	Readability assessment aims to automatically classify texts based on readers’ reading levels.
Approach:	They propose a hybrid automatic readability assessment model using prompts to improve deep feature representations and an orthogonal projection layer to fuse both deep and linguistic features.
Outcome:	The proposed model outperforms state-of-the-art models over four English and two Chinese corpora and demonstrates that it is more efficient than existing models.

Copied to clipboard

Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, Felix Juefei-Xu, Foutse Khomh, Osamu Yoshie, Qingyu Chen, Douglas Teodoro, Nan Liu, Randy Goebel, Lei Ma, Edison Marrese-Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, Irene Li

Multilinguality

Challenge:	Existing large language model evaluation benchmarks focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities.
Approach:	They propose a comprehensive benchmark covering 29 languages, built on an English benchmark.
Outcome:	The MMLU-ProX is a comprehensive benchmark covering 29 languages, built on an English benchmark.

Me-Agent: A Personalized Mobile Agent with Two-Level User Habit Learning for Enhanced Interaction (2026.findings-acl)

Copied to clipboard

Shuoxin Wang, Chang Liu, Gowen Loo, Lifan Zheng, Kaiwen Wei, Huanqian Yan, Xinyi Zeng, Jingyuan Zhang, Yu Tian

Dict-BERT: Enhancing Language Model Pre-training with Dictionary (2022.findings-acl)

Challenge:	Existing Large Language Model (LLM)-based mobile agents follow explicit user instructions without personalized needs.
Approach:	They propose a user preference learning strategy enhanced with a Personal Reward Model to improve personalization performance.
Outcome:	The proposed agent achieves state-of-the-art performance while maintaining competitive instruction execution performance.

Copied to clipboard

Wenhao Yu, Chenguang Zhu, Yuwei Fang, Donghan Yu, Shuohang Wang, Yichong Xu, Michael Zeng, Meng Jiang

Instructions are all you need: Self-supervised Reinforcement Learning for Instruction Following (2026.acl-long)

Challenge:	Pre-trained language models (PLMs) capture word semantics in different contexts, hence the embeddings of rare words on the tail are poorly optimized.
Approach:	They propose to leverage definitions of rare words in dictionaries to enhance language model pre-training by leveraging dictionary definitions.
Outcome:	The proposed model improves understanding of rare words and boosts performance on various NLP downstream tasks.

Copied to clipboard

Qingyu Ren, Qianyu He, Powei Chang, Jie Zeng, Zeye Sun, Fei Yu, Jiaqing Liang, Yanghua Xiao

Fin-STAR: Structure-as-Semantics to Resolve Implicitness in Financial Retrieval (2026.findings-acl)

Challenge:	Existing reinforcement learning approaches suffer from dependency on external supervision and sparse reward signals from multi-constraint tasks.
Approach:	They propose a self-supervised reinforcement learning framework that eliminates dependency on external supervision by deriving reward signals directly from instructions and generating pseudo-labels for reward model training.
Outcome:	The proposed framework achieves strong improvements across 3 in-domain and 5 out-of-domain datasets while maintaining computational efficiency.

Copied to clipboard

Yu Zou, Yan Chen, Lida He, Qi Zhou, Xiaorui Zhou, Aixi Zhong, Yi Wang, Wei Li, Qingyu Wang, Jiatao Li, Wei Gong, Jialei Zeng, Jingmei Zhao, Ke Jiang, Qing Li

MinT: Boosting Generalization in Mathematical Reasoning via Multi-view Fine-tuning (2024.lrec-main)

Challenge:	Existing Retrieval-Augmented Generation systems treat structure as a physical navigational skeleton rather than intrinsic semantic knowledge.
Approach:	They propose a framework that redefining hierarchy as intrinsic semantics and uses snippets to enrich hierarchical lineage.
Outcome:	The proposed framework outperforms state-of-the-art hierarchical and graph-based benchmarks on FinTierQA Gold.

Copied to clipboard

Zhenwen Liang, Dian Yu, Xiaoman Pan, Wenlin Yao, Qingkai Zeng, Xiangliang Zhang, Dong Yu

UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervision (2026.acl-long)

Challenge:	Existing methods focus on specializing LMs in mathematical reasoning and rely on knowledge distillation.
Approach:	They propose a multi-view fine-tuning method that exploits existing mathematical problem datasets with diverse annotation styles.
Outcome:	The proposed method outperforms existing methods that rely heavily on LLM teachers . it grants models generalization ability across views and datasets, and the capability to learn from inaccurate or incomplete data.

Copied to clipboard

Zhen Fang, Ruiyan Han, XinYu Sun, Yuchen Ma, Ziheng Wang, Yu Zeng, Zehui Chen, Lin Chen, Wenxuan Huang, Wei-Jie Xu, Yi Cao, Feng Zhao

Detecting AI-Generated Content on Social Media with Multi-modal Language Models (2026.acl-industry)

Challenge:	Unified Multimodal Models have achieved remarkable success in cross-modal comprehension, but a gap persists in their ability to translate internal knowledge into faithful and controllable synthesis.
Approach:	They propose a self-improvement framework that partitions a single UMM into three collaborative roles: Proposer, Solver, and Judge.
Outcome:	The proposed framework improves on TIIF, DPG, CompBench and UniCycle benchmarks.

Copied to clipboard

Chenyang Yang, Shen Yan, Yibo Yang, Litao Hu, Yuchen Liu, Yuan Zeng, Hanchao Yu, Yinan Zhu, Sumedha Singla, Brian Vanover, Huijun Qian, Zihao Wang, Fujun Liu, Aashu Singh, Jianyu Wang, Xuewen Zhang

DeepSieve: Information Sieving via LLM-as-a-Knowledge-Router (2026.findings-eacl)

Challenge:	Existing methods for AI-generated content detection face poor generalization to newer models, reliance on single modalities, and lack of interpretable explanations.
Approach:	They propose a model that curates diverse social media data and trains a vision-language model for detection and explanation.
Outcome:	The proposed model achieves state-of-the-art detection performance on public benchmarks and observes positive downstream impacts on user engagement.

Copied to clipboard

Minghao Guo, Qingcheng Zeng, Xujiang Zhao, Yanchi Liu, Wenchao Yu, Mengnan Du, Haifeng Chen, Wei Cheng

On the Generation of Medical Dialogs for COVID-19 (2021.acl-short)

Challenge:	Existing RAG methods lack fine-grained control over query and source sides, resulting in noisy retrieval and shallow reasoning.
Approach:	They propose an agentic RAG framework that integrates information sieving via LLM-as-a-knowledge-router.
Outcome:	Experiments on multi-hop QA tasks across heterogeneous sources demonstrate improved reasoning depth, retrieval precision, and interpretability over conventional approaches.

Copied to clipboard

Meng Zhou, Zechen Li, Bowen Tan, Guangtao Zeng, Wenmian Yang, Xuehai He, Zeqian Ju, Subrato Chakravorty, Shu Chen, Xingyi Yang, Yichen Zhang, Qingyang Wu, Zhou Yu, Kun Xu, Eric Xing, Pengtao Xie

Dialogue Systems & Conversational Agents

Challenge:	under the pandemic of COVID-19, people experiencing COVI D19-related symptoms have a pressing need to consult doctors.
Approach:	They develop a medical dialog system that can provide COVID19-related consultations . they use two dialog datasets containing conversations between doctors and patients .
Outcome:	The proposed system can provide COVID19-related consultations, but is too small compared with general-domain dialog datasets.

Joint Optimization of Training Data and Policy in RLHF (2026.findings-acl)

Copied to clipboard

Zhuohao Yu, Jiali Zeng, Weizheng Gu, Mengyuan Sun, Yidong Wang, Fandong Meng, Jie Zhou, Shikun Zhang, Wei Ye

Challenge:	JODP optimizes policies on fixed training inputs, limiting the diversity of learning signals.
Approach:	They propose a framework where policy generates improved variants of training problems to enhance its own learning.
Outcome:	The proposed framework improves on safety alignment tasks by allowing 4B models to reach 8B model performance with less than 1% additional computational overhead.

CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution (2026.acl-long)

Copied to clipboard

Xiangxi Zheng, Kuang He, Jiayi Hu, Ping Yu, Rui Yan, Yuan Yao, Peng Hou, Anxiang Zeng, Alex Jinpeng Wang

Enhancing Automatic Readability Assessment with Pre-training and Soft Labels for Ordinal Regression (2022.findings-emnlp)

Challenge:	Existing approaches to chart-to-code generation are constrained by data-centric limitations . authors present a new framework that redesigns both training and alignment data .
Approach:	They propose a data-centric framework that redesigns both training and alignment data for chart-to-code generation.
Outcome:	The proposed framework outperforms open-source baselines and is competitive with GPT-5.

Copied to clipboard

Jinshan Zeng, Yudong Xie, Xianglong Yu, John Lee, Ding-Xuan Zhou

Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Models (2025.coling-main)

Challenge:	Existing models do not exploit ordinal nature of difficulty grades and make little effort for initialization to facilitate fine-tuning.
Approach:	They propose a readability assessment task that assigns a difficulty grade to a text . they use ordinal regression and pairwise relative text difficulty to train the model .
Outcome:	The proposed model outperforms competitive neural models and statistical classifiers on most datasets.

Copied to clipboard

Hongchuan Zeng, Senyu Han, Lu Chen, Kai Yu

RHO: Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding (2023.findings-acl)

Challenge:	Recent studies suggest that large language models can transfer skills learned in one language to others, but internal mechanisms behind this ability remain unclear.
Approach:	They find that LLMs map semantically identical inputs from different languages into a common semantic latent space that allows for consistent processing across languages.
Outcome:	The findings highlight the structural evolution of multilingual models during training and scaling up.

Copied to clipboard

Ziwei Ji, Zihan Liu, Nayeon Lee, Tiezheng Yu, Bryan Wilie, Min Zeng, Pascale Fung

Structured Data in NLP

Challenge:	Existing knowledge-grounded dialogue systems generate accurate and informative responses, but they are prone to hallucination problems.
Approach:	They propose a method to generate hallucinated responses using knowledge graphs . they propose local knowledge grounding to combine textual embeddings with corresponding KG embeddments . a global knowledge ground technique is also proposed to equip RHO with multi-hop reasoning abilities .
Outcome:	The proposed approach outperforms state-of-the-art methods on automatic and human evaluation by a large margin.

TabDSR: Decompose, Sanitize, and Reason for Complex Numerical Reasoning in Tabular Data (2025.findings-emnlp)

Copied to clipboard

Changjiang Jiang, Fengchang Yu, Haihua Chen, Wei Lu, Jin Zeng

Uncertainty Unveiled: Can Exposure to More In-context Examples Mitigate Uncertainty for Large Language Models? (2025.findings-acl)

Challenge:	Large language models often underperform due to complex queries, noisy data, and limited numerical capabilities.
Approach:	They propose a framework that integrates seamlessly with mainstream LLMs to improve tabular reasoning.
Outcome:	The proposed framework outperforms existing methods in state-of-the-art analysis.

Copied to clipboard

Yifei Wang, Yu Sheng, Linjing Li, Daniel Dajun Zeng

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models (2026.acl-long)

Challenge:	Recent advances in handling long sequences have unlocked new possibilities for long-context in-contact learning (ICL).
Approach:	They investigate how increased examples influence predictive uncertainty . they quantify uncertainty across different “shot” configurations and focus on EU .
Outcome:	The proposed model reduces uncertainty in simple and complex tasks by injecting task-specific knowledge.

Copied to clipboard

Xin Cheng, Wangding Zeng, Damai Dai, Qinyu Chen, Bingxuan Wang, Zhenda Xie, Kezhao Huang, Xingkai Yu, Zhewen Hao, Han Zhang, Yu-Kun Li, Huishuai Zhang, Dongyan Zhao, Wenfeng Liang

DiffER: Diffusion Entity-Relation Modeling for Reversal Curse in Diffusion Large Language Models (2026.findings-acl)

Challenge:	Mixture-of-Experts (MoE) scales capacity via conditional computation, but lacks knowledge lookup primitive.
Approach:	They propose a conditional memory instantiated via Deep Sparse Embedding (DSE) they propose 'u-shaped scaling law' that identifies optimal balance between MoE experts and DSE memory .
Outcome:	The proposed model outperforms an iso-parameter and isoFLOPs MoE baseline across knowledge and reasoning benchmarks and is infrastructure-efficient.

Copied to clipboard

Shaokai He, Kaiwen Wei, Xinyi Zeng, Xiang Chen, Xue Yang, Zhenyang Li, Jiang Zhong, Yu Tian

EFSA: Towards Event-Level Financial Sentiment Analysis (2024.acl-long)

Challenge:	Existing large language models exhibit unidirectional behavior when processing bidirectional relationships . authors propose a solution to alleviate the reversal curse in Diffusion LLMs .
Approach:	They propose a model that addresses the "reversal curse" of bidirectional behavior in large language models . they propose 'entity-aware training' and balanced data construction to alleviate asymmetry and missing relations .
Outcome:	The proposed model alleviates the "reversal curse" in Diffusion LLMs . the proposed model employs whole-entity masking to mitigate entity fragmentation .

Copied to clipboard

Tianyu Chen, Yiming Zhang, Guoxin Yu, Dapeng Zhang, Li Zeng, Qing He, Xiang Ao

Sentiment Analysis

Challenge:	a large-scale Chinese dataset contains 12,160 news articles and 13,725 quintuples . a four-hop Chain-of-Thought LLM-based approach is devised for this task .
Approach:	They propose to extend financial sentiment analysis to event-level since events usually serve as the subject of the sentiment in financial text.
Outcome:	The proposed method can reach the current state-of-the-art on a large-scale Chinese dataset.

Order Matters: Investigate the Position Bias in Multi-constraint Instruction Following (2025.findings-acl)

Copied to clipboard

Jie Zeng, Qianyu He, Qingyu Ren, Jiaqing Liang, Weikang Zhou, Zeye Sun, Fei Yu, Yanghua Xiao

Challenge:	Existing large language models struggle to follow multi-constraint instructions in real-world applications.
Approach:	They propose to quantify the difficulty distribution of constraints by a novel Difficulty Distribution Index (CDDI) they find that LLMs are more performant when presented with constraints in a “hard-to-easy” order.
Outcome:	The proposed model is more performant when presented with constraints in a “hard-to-easy” order, compared with existing models with different architectures and sizes of parameters.

A Technical Question Answering System with Transfer Learning (2020.emnlp-demos)

Copied to clipboard

Wenhao Yu, Lingfei Wu, Yu Deng, Ruchi Mahindru, Qingkai Zeng, Sinem Guven, Meng Jiang

Natural Language Interfaces

Challenge:	StackOverflow and AskUbuntu are popular open forum communities for technical question-answering, but it is expensive for human experts to provide timely and helpful responses.
Approach:	They develop a system that automatically responds to questions from a siamese ALBERT network based on previously answered questions .
Outcome:	The proposed system responds automatically to questions based on answers from previous users.

A Survey on Natural Language Counterfactual Generation (2024.findings-emnlp)

Copied to clipboard

Yongjie Wang, Xiaoqi Qiu, Yu Yue, Xu Guo, Zhiwei Zeng, Yuhong Feng, Zhiqi Shen

Challenge:	Recent advances in NLP are driven by a variety of Large Language Models (LLMs), such as GPT-3 (175B) and PaLM (540B).
Approach:	They propose a taxonomy that categorizes the methods into four groups and summarizes the metrics for evaluating the generation quality.
Outcome:	The proposed taxonomy categorizes the generation methods into four groups and summarizes the metrics for evaluating the quality.

Root Defense Strategies: Ensuring Safety of LLM at the Decoding Level (2025.acl-long)

Copied to clipboard

Xinyi Zeng, Yuying Shang, Jiawei Chen, Jingyuan Zhang, Yu Tian

Marco-o1 v2: Towards Widening The Distillation Bottleneck for Reasoning Models (2025.acl-long)

Challenge:	Existing methods to detect harmful outputs from prefill-level lacks utilization of the model’s decoding outputs, leading to relatively lower effectiveness and robustness.
Approach:	They propose a robust decoding mechanism that corrects harmful queries directly rather than rejecting them outright.
Outcome:	The proposed model improves model security without compromising reasoning speed.

Copied to clipboard

Huifeng Yin, Yu Zhao, Minghao Wu, Xuanfan Ni, Bo Zeng, Huaiyu.wh Huaiyu.wh, Tianqi Shi, Liangying Shao, Chenyang Lyu, Longyue Wang, Weihua Luo, Kaifu Zhang

KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering (2022.acl-long)

Challenge:	Recent efforts to distill large reasoning models into smaller lightweight models have shown competitive performances.
Approach:	They propose to distill long Chain-of-Thought data to improve SFT and RL methods by constructing data from scratch using Monte Carlo Tree Search.
Outcome:	The proposed method significantly improves reasoning performance on various benchmarks such as math (GSM8K, MATH, AIME).

Copied to clipboard

Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, Michael Zeng

Task Compass: Scaling Multi-task Pre-training with Task Prefix (2022.findings-emnlp)

Challenge:	Open-Domain Question Answering (ODQA) models typically include a retrieving module and a reading module.
Approach:	They propose a new open-domain question-answering framework that uses a knowledge-enhanced version of FiD to improve the approach.
Outcome:	The proposed model improves on ODQA benchmark datasets with less than 40% computation cost.

Copied to clipboard

Zhuosheng Zhang, Shuohang Wang, Yichong Xu, Yuwei Fang, Wenhao Yu, Yang Liu, Hai Zhao, Chenguang Zhu, Michael Zeng

Evaluating Generalization Capability of Language Models across Abductive, Deductive and Inductive Logical Reasoning (2025.coling-main)

Challenge:	Existing studies show that multi-task learning with large-scale supervised tasks suffers from negative effects across tasks.
Approach:	They propose a task prefix guided multi-task pre-training framework to explore the relationships among tasks.
Outcome:	The proposed model can be used as a foundation backbone for a wide range of tasks and as augmentation tool for data augmentation with complementary tasks.

Copied to clipboard

Yu Sheng, Wanting Wen, Linjing Li, Daniel Zeng

Tri-Train: Automatic Pre-Fine Tuning between Pre-Training and Fine-Tuning for SciNER (2020.findings-emnlp)

Challenge:	Recent research in language models (LMs) have demonstrated remarkable performance on many natural language tasks, yet to what extent LMs possess the capability of generalizing to unseen logical rules remains unclear.
Approach:	They propose to use a dataset to assess the generalization capabilities of LMs on ADI reasoning to assess their generalization abilities.
Outcome:	The proposed dataset shows that LMs perform poorly on ADI reasoning tasks and lacks generalization capabilities.

Copied to clipboard

Qingkai Zeng, Wenhao Yu, Mengxia Yu, Tianwen Jiang, Tim Weninger, Meng Jiang

Activation-Guided Local Editing for Jailbreaking Attacks (2026.acl-long)

Challenge:	Pre-training a language model by self-supervised tasks on huge datasets and fine-tuning with small labelled data are often inadequate for scientific NER tasks.
Approach:	They propose to introduce a "pre-fine tuning" step between pre-training and fine-tuning to construct a corpus by selecting sentences from unlabeled documents that are the most relevant with labelled training data.
Outcome:	The proposed approach improves on seven benchmarks on the performance of the proposed model on labelled datasets.

Copied to clipboard

Jiecong Wang, Haoran Li, Hao Peng, Ziqian Zeng, Zihao Wang, Haohua Du, Zhengtao Yu

Robustness in NLP

Challenge:	Existing methods for jailbreaking Large Language Models (LLMs) are limited and produce incoherent or unreadable inputs.
Approach:	They propose a two-stage framework that performs a one-shot, scenario-based generation of context and rephrases the original malicious query to obscure its harmful intent.
Outcome:	The proposed framework achieves state-of-the-art Attack Success Rate, with gains of up to 37.74% over the strongest baseline, and excellent transferability to black-box and large-scale models.

VisualRWKV: Exploring Recurrent Neural Networks for Visual Language Models (2025.coling-main)

Copied to clipboard

Haowen Hou, Peigen Zeng, Fei Ma, Fei Richard Yu

Attention Calibration for Transformer in Neural Machine Translation (2021.acl-long)

Challenge:	Visual Language Models (VLMs) have been gaining popularity with large language models, but few attempts have been made to incorporate efficient linear Recurrent Neural Networks (RNNs) into VLMs.
Approach:	They propose a linear RNN model with a data-dependent recurrence and sandwich prompts to enhance modeling capabilities and a 2D image scanning mechanism to enrich the processing of visual sequences.
Outcome:	The proposed model achieves competitive performance compared to Transformer-based models on various benchmarks.

Copied to clipboard

Yu Lu, Jiali Zeng, Jiajun Zhang, Shuangzhi Wu, Mu Li

TiMem: Temporal-Hierarchical Memory Consolidation for Long-Horizon Conversational Agents (2026.findings-acl)

Challenge:	Attention mechanisms have been ubiquitous in neural machine translation (NMT) however, many studies doubt whether highlyattended inputs have a large impact on the model outputs.
Approach:	They propose to introduce a mask perturbation model that automatically evaluates each input’s contribution to the model outputs.
Outcome:	The proposed model is more uniform at lower layers while more concentrated on the specific inputs at higher layers.

Copied to clipboard

Kai Li, Xuanqing Yu, Ziyi Ni, Yi Zeng, Yao Xu, Zheqing Zhang, Xin Li, Jitao Sang, Xiaogang Duan, Xuelei Wang, Chengbao Liu, Jie Tan

Dialogue Systems & Conversational Agents

Challenge:	Existing memory frameworks provide limited support for temporally structured information across hierarchical levels, leading to fragmented memories and unstable long-horizon personalization.
Approach:	They propose a temporal–hierarchical memory framework that organizes conversations through a Temporal Memory Tree.
Outcome:	The proposed framework outperforms baselines while reducing the recalled memory length by 52.20%.

FreeEval: A Modular Framework for Trustworthy and Efficient Evaluation of Large Language Models (2024.emnlp-demo)

Copied to clipboard

Zhuohao Yu, Chang Gao, Wenjin Yao, Yidong Wang, Zhengran Zeng, Wei Ye, Jindong Wang, Yue Zhang, Shikun Zhang

Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models (2023.emnlp-main)

Challenge:	Large language models (LLMs) have revolutionized natural language processing with impressive performance across various tasks.
Approach:	They propose a framework for automated evaluations of large language models . they open-source their code at https://github.com/WisdomShell/FreeEval .
Outcome:	The framework is open-source and can be used to develop and validate new evaluation methods.

Copied to clipboard

Yifan Hou, Jiaoda Li, Yu Fei, Alessandro Stolfo, Wangchunshu Zhou, Guangtao Zeng, Antoine Bosselut, Mrinmaya Sachan