Papers by Lin Ye

CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios (2025.emnlp-main)

Challenge:	Knowledge distillation is an effective method for model acceleration and compression.
Approach:	They propose to use parameters to distill knowledge from large neural networks to small ones . they propose to do this by using a parameter generator to transfer the knowledge to a small neural network .
Outcome:	The proposed method learns a small network 1.88 2.94x faster than the large network but with competitive BLEU points.

Copied to clipboard

Shiting Huang, Zhen Fang, Zehui Chen, Siyu Yuan, Junjie Ye, Yu Zeng, Lin Chen, Qi Mao, Feng Zhao

Bag of Tricks for Optimizing Transformer Efficiency (2021.findings-emnlp)

Challenge:	a number of tools are used to perform complex tasks, but the tool utilization process can cause errors.
Approach:	They propose a critique evaluation benchmark for tool learning that analyzes function-calling errors on tool evaluation benchmarks.
Outcome:	The proposed critique evaluation benchmark holds diverse tool-use errors with varying complexities, which better reflects real-world scenarios.

Copied to clipboard

Ye Lin, Yanyang Li, Tong Xiao, Jingbo Zhu

FOLIO: Natural Language Reasoning with First-Order Logic (2024.emnlp-main)

Challenge:	Improving Transformer efficiency has become increasingly attractive in recent years.
Approach:	They propose to combine pruning, quantization, new architectures and training strategies to improve Transformer efficiency.
Outcome:	The proposed methods improve the inference efficiency of a strong Transformer system by 3.80x on CPU and 2.52x on GPU.

Copied to clipboard

Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, Lucy Sun, Alexander Wardle-Solano, Hannah Szabó, Ekaterina Zubova, Matthew Burtell, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kasai, Tao Yu, Rui Zhang, Alexander Fabbri, Wojciech Kryscinski, Semih Yavuz, Ye Liu, Xi Lin, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Rex Ying, Arman Cohan, Dragomir Radev

Argue with Me Tersely: Towards Sentence-Level Counter-Argument Generation (2023.emnlp-main)

Challenge:	Existing benchmarks for logical reasoning in large language models lack language naturalness or limited complexity.
Approach:	They propose to use first-order logic annotations to evaluate logical reasoning capabilities of large language models.
Outcome:	The proposed dataset evaluates the FOL reasoning ability of supervised fine-tuning on medium-sized language models.

Copied to clipboard

Jiayu Lin, Rong Ye, Meng Han, Qi Zhang, Ruofei Lai, Xinyu Zhang, Zhao Cao, Xuanjing Huang, Zhongyu Wei

Text Generation

Challenge:	Existing work describes paragraph-level counter-argument generation task as paragraph-based . however, sentence-level generation can be quite different due to its unique constraints and brevity-focused challenges.
Approach:	They propose a benchmark framework for sentence-level counter-argument generation . they use an annotated debate forum dataset to generate high-quality counter-argments .
Outcome:	The proposed framework and evaluator are competitive in counter-argument generation tasks.

Omni-Chart-600K: A Comprehensive Dataset of Chart Types for Chart Understanding (2025.findings-naacl)

Copied to clipboard

Shulei Wang, Shuai Yang, Wang Lin, Zirun Guo, Sihang Cai, Hai Huang, Ye Wang, Jingyuan Chen, Tao Jin

ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching (2026.findings-acl)

Challenge:	Existing chart-related training methods lack capabilities in information extraction, mathematical reasoning, and understanding of multiple chart types.
Approach:	They propose a two-stage training strategy and method for jointly training a vision encoder tailored for multi-type charts to address the deficiencies in chart types and limited scope of chart tasks in existing datasets.
Outcome:	The proposed dataset includes 21 diverse chart types and tasks, including data retrieval and mathematical reasoning.

Copied to clipboard

Han Zhu, Wei Kang, Liyong Guo, Zengwei Yao, Fangjun Kuang, Weiji Zhuang, Zhaoqing Li, Zhifeng Han, Dong Zhang, Xin Zhang, Xingchen Song, Lingxuan Ye, Long Lin, Daniel Povey

Dialogue Systems & Conversational Agents

Challenge:	Existing autoregressive models for dialogue generation suffer from high latency and stability issues.
Approach:	They propose a non-autoregressive (NAR) zero-shot spoken dialogue generation model based on flow-matching.
Outcome:	The proposed model outperforms existing models in speech generation due to poor speech intelligibility and turn-taking precision.

GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment (2026.acl-long)

Copied to clipboard

Jiwei Tang, Zhicheng Zhang, Shunlong Wu, Jingheng Ye, Lichen Bai, Zitai Wang, Tingwei Lu, Lin Hai, Yiming Zhao, Hai-Tao Zheng, Hong-Gee Kim

RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation (2024.emnlp-demo)

Challenge:	Large Language Models (LLMs) have achieved remarkable performance across NLP tasks . however, in long-context scenarios, they face high computational cost and information redundancy.
Approach:	They propose an encoder-decoder context compression framework that generates a compact sequence of soft tokens for downstream tasks.
Outcome:	Experiments show that GMSA outperforms baselines on multiple long-context question answering and summarization benchmarks while maintaining low end-to-end latency.

Copied to clipboard

Qinyu Luo, Yining Ye, Shihao Liang, Zhong Zhang, Yujia Qin, Yaxi Lu, Yesai Wu, Xin Cong, Yankai Lin, Yingli Zhang, Xiaoyin Che, Zhiyuan Liu, Maosong Sun

A Simple and Effective Approach to Robust Unsupervised Bilingual Dictionary Induction (2020.coling-main)

Challenge:	Xia et al., 2018) demonstrate that a large language model can generate and maintain high-quality code documentation.
Approach:	They propose a large language model powered open-source framework for generating, maintaining, and updating code documentation.
Outcome:	The proposed framework generates high-quality documentation for the entire project.

Copied to clipboard

Yanyang Li, Yingfeng Luo, Ye Lin, Quan Du, Huizhen Wang, Shujian Huang, Tong Xiao, Jingbo Zhu

Coreferential Reasoning Learning for Language Representation (2020.emnlp-main)

Challenge:	Recent work has questioned the robustness of unsupervised bilingual dictionary induction methods on distant language pairs.
Approach:	They propose an iterative dimension reduction method to bridge this gap . they propose a method that initializes and self-learning and inducing a dictionary .
Outcome:	The proposed method achieves 13.64 55.53% accuracy between English and four distant languages.

Copied to clipboard

Deming Ye, Yankai Lin, Jiaju Du, Zhenghao Liu, Peng Li, Maosong Sun, Zhiyuan Liu

Packed Levitated Marker for Entity and Relation Extraction (2022.acl-long)

Challenge:	Existing language representation models cannot explicitly handle coreference, which is essential to the coherent understanding of the whole discourse.
Approach:	They propose a language representation model that captures coreferential relations in context.
Outcome:	The proposed model can achieve significant improvements on downstream NLP tasks while maintaining comparable performance to baseline models on other common NLP task.

Copied to clipboard

Deming Ye, Yankai Lin, Peng Li, Maosong Sun

Information Extraction & Text Mining

Challenge:	Existing work on entity and relation extraction ignores the interrelation between spans . a novel approach to extract better span representations from pre-trained languages is needed .
Approach:	They propose a span representation approach that packs Levitated Markers to consider interrelation between spans.
Outcome:	The proposed model improves on baselines on six NER benchmarks and achieves a 4.1%-4.3% strict relation F1 improvement with higher speed over previous state-of-the-art models.

TAVT: Towards Transferable Audio-Visual Text Generation (2023.acl-long)

Copied to clipboard

Wang Lin, Tao Jin, Wenwen Pan, Linjun Li, Xize Cheng, Ye Wang, Zhou Zhao

Temporal Scaling Law for Large Language Models (2025.emnlp-main)

Challenge:	Existing transfer learning techniques focus on uni-modal analysis and lack consideration of multi-modal content and cross-modal relation.
Approach:	They propose a transferable audio-visual text generation framework that incorporates two components: Audio-Visual Meta-Mapper and Dual Counterfactual Contrastive Learning.
Outcome:	The proposed framework outperforms the state-of-the-art methods across multiple domains and modal settings.

Copied to clipboard

Yizhe Xiong, Xiansheng Chen, Xin Ye, Hui Chen, Zijia Lin, Haoran Lian, Zhenpeng Su, Wei Huang, Jianwei Niu, Jungong Han, Guiguang Ding

NOVA-63: Native Omni-lingual Versatile Assessments of 63 Disciplines (2025.emnlp-main)

Challenge:	Existing studies have found that the test loss of LLMs scales as power-laws with model size, computational budget, and dataset size.
Approach:	They propose a concept of Temporal Scaling Law to study test loss of LLMs . they break down test loss into fine-grained token positions and develop a dynamic hyperbolic-law .
Outcome:	The proposed model predicts the test loss of LLMs as the training steps scale up.

Copied to clipboard

Jinyang Zhang, Kexin Yang, Yu Wan, Muyang Ye, Baosong Yang, Fei Huang, Junyang Lin, Dayiheng Liu

Multilinguality

Challenge:	Existing multilingual benchmarks show severe drawbacks, such as overly translated content, the absence of difficulty control, and disciplinary imbalance, making the benchmarking process unreliable and showing low convincingness.
Approach:	They propose a multilingual benchmark that integrates LLM-assisted formatting, expert quality verification, and multi-level difficulty screening to provide a comprehensive, difficult multilingual assessment.
Outcome:	The proposed benchmark features 93,536 questions sourced from native speakers across 14 languages and 63 academic disciplines.

Weakly-Supervised Spoken Video Grounding via Semantic Interaction Learning (2023.acl-long)

Copied to clipboard

Ye Wang, Wang Lin, Shengyu Zhang, Tao Jin, Linjun Li, Xize Cheng, Zhou Zhao

Multi-Path Transformer is Better: A Case Study on Neural Machine Translation (2022.findings-emnlp)

Challenge:	Recent work on spoken video grounding challenges extracting semantic information from speech . previous studies focused on textual queries, but recent work focuses on spoken queries .
Approach:	They propose a framework for weakly-supervised spoken video grounding to represent cross-modal semantics without expensive temporal annotations.
Outcome:	The proposed framework is more efficient than existing methods.

Copied to clipboard

Ye Lin, Shuhan Zhou, Yanyang Li, Anxiang Ma, Tong Xiao, Jingbo Zhu

MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering (2025.findings-acl)

Challenge:	Extensive experiments on 12 WMT tasks show that shallower multi-path models can achieve similar or even better performance than the deeper model.
Approach:	They propose to use a parameter-efficient multi-path structure to fuse features extracted from different paths to achieve better performance.
Outcome:	The proposed model can achieve better performance with the same number of parameters than the deeper model.

Copied to clipboard

Jingqun Tang, Qi Liu, Yongjie Ye, Jinghui Lu, Shu Wei, An-Lan Wang, Chunhui Lin, Hao Feng, Zhen Zhao, Yanjie Wang, Yuliang Liu, Hao Liu, Xiang Bai, Can Huang

Multilinguality

Challenge:	Text-Centric Visual Question Answering (TEC-VQA) is a text-centric visual task understanding tool.
Approach:	They introduce a benchmark that features human expert annotations across 9 languages . they prioritize the text in question-answer pairs while disregarding visual text in images .
Outcome:	The proposed benchmarks prioritize the text in question-answer pairs while disregarding visual text in images.

AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios (2025.naacl-long)

Copied to clipboard

Xinyi Mou, Jingcong Liang, Jiayu Lin, Xinnong Zhang, Xiawei Liu, Shiyue Yang, Rong Ye, Lei Chen, Haoyu Kuang, Xuanjing Huang, Zhongyu Wei

Linguistics & Cognitive NLP

Challenge:	Large language models are increasingly employed to empower autonomous agents to simulate human behavior.
Approach:	They propose to evaluate LLM-driven agents through multi-turn interactions using a bottom-up approach to create diverse social scenarios constructed from extensive scripts.
Outcome:	The proposed model evaluates LLM-driven agents through multi-turn interactions emphasizing goal completion and implicit reasoning.

LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation (2020.acl-demos)

Copied to clipboard

Dong-Ho Lee, Rahul Khanna, Bill Yuchen Lin, Seyeon Lee, Qinyuan Ye, Elizabeth Boschee, Leonardo Neves, Xiang Ren

AdamMeme: Adaptively Probe the Reasoning Capacity of Multimodal Large Language Models on Harmfulness (2025.acl-long)

Challenge:	Existing frameworks for sequence labeling and classification require massive human effort and labeling data is limited.
Approach:	They propose a web-based, Label-Efficient AnnotatioN framework that allows an annotator to provide the needed labels for a task and can capture explanations for each labeling decision.
Outcome:	The proposed framework surpasses baseline F1 scores by 5-10 percentage points while using 2X times fewer labeled instances.

Copied to clipboard

Zixin Chen, Hongzhan Lin, Kaixin Li, Ziyang Luo, Zhen Ye, Guang Chen, Zhiyong Huang, Jing Ma

A Novel Matching Paradigm: Unified Generative and Discriminative LLM with Prompt Compression for Relevance Learning (2026.acl-industry)

Challenge:	Existing models that assess mLLMs on harmful meme understanding are inaccurate and lack accuracy.
Approach:	They propose a framework that adaptively probes the reasoning capabilities of mLLMs . their framework systematically reveals the varying performance of different target mllms a .
Outcome:	The proposed framework systematically reveals the performance of different target mLLMs.

Copied to clipboard

Guoliang Zhao, Zixin Cui, Chao Ye, Dengwu He, Fei Huang, Yubo Liu, Shuanglong Li, Tzungren Kuo, Bin Ding, Shuang Zhang, null KunhongZhu, Zhi Guo, Liu Lin

TrendSim: Simulating Trending Topics in Social Media Under Poisoning Attacks with LLM-based Multi-agent System (2025.findings-naacl)

Challenge:	Existing approaches to matching use Large Language Models as feature extractors, underutilizing their full modeling capabilities.
Approach:	They propose a matching paradigm that integrates two-tower, single-towing, and generative tasks within a unified LLM framework via attention-mask partitioning.
Outcome:	The proposed model achieves superior performance and strong practical value in an industrial search engine.

Copied to clipboard

Zeyu Zhang, Jianxun Lian, Chen Ma, Yaning Qu, Ye Luo, Lei Wang, Rui Li, Xu Chen, Yankai Lin, Le Wu, Xing Xie, Ji-Rong Wen

Robustness in NLP

Challenge:	Trending topics bring in a new channel for poisoning attacks, resulting in negative impacts on society.
Approach:	They propose an LLM-based multi-agent system to simulate trending topics in social media . they propose a time-aware interaction mechanism, centralized message dissemination, and an interactive system .
Outcome:	The proposed system simulates trending topics under poisoning attacks on social media platforms.

Video-LLaVA: Learning United Visual Representation by Alignment Before Projection (2024.emnlp-main)

Copied to clipboard

Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, Li Yuan

TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference (2021.naacl-main)

Challenge:	Existing approaches to visual-language understanding lack unified tokenization for images and videos . lack of unified visual representations makes it difficult to learn multi-modal interactions from poor projection layers.
Approach:	They propose to unify visual representation into the language feature space to advance the foundational LLM towards a unified LVLM.
Outcome:	The proposed model outperforms Video-ChatGPT on image benchmarks and on 9 image benchmark benchmarks.

Copied to clipboard

Deming Ye, Yankai Lin, Yufei Huang, Maosong Sun

Act-Adaptive Margin: Dynamically Calibrating Reward Models for Subjective Ambiguity (2026.acl-long)

Challenge:	Existing pre-trained language models (PLMs) are expensive in inference, making them impractical in resource-limited real-world applications.
Approach:	They propose a dynamic token reduction approach to accelerate PLMs' inference by adapting the layer number of each token to avoid redundant calculation.
Outcome:	The proposed approach speeds up BERT by 2-5 times and improves performance in long-text tasks with less computation.

Copied to clipboard

Feiteng Fang, Dingwei Chen, Xiang Huang, Ting-En Lin, Yuchuan Wu, Xiong Liu, Jing Ye, Ziqiang Liu, Haonan Zhang, Liang Zhu, Hamid Alinejad-Rokny, Min Yang, Yongbin Li

Challenge:	Existing approaches to reward modeling in reinforcement learning tasks are limited when dealing with ambiguous preferences.
Approach:	They propose to use AAM to dynamically calibrate preference margins using the Bradley-Terry model's internal parameter knowledge to improve reward modeling in subjective tasks.
Outcome:	The proposed approach improves reward modeling by dynamically calibrating preference margins using the model’s internal parameter knowledge.

MobileNMT: Enabling Translation in 15MB and 30ms (2023.acl-industry)

Copied to clipboard

Ye Lin, Xiaohui Wang, Zhexi Zhang, Mingxuan Wang, Tong Xiao, Jingbo Zhu

Multilinguality

Challenge:	Existing work on NMT models is limited in storage, memory, computation and power consumption.
Approach:	They propose a mobile machine translation system that can translate in 15MB and 30ms on devices.
Outcome:	The proposed system can translate in 15MB and 30ms on mobile devices.

TRIGO: Benchmarking Formal Mathematical Proof Reduction for Generative Language Models (2023.emnlp-main)

Copied to clipboard

Jing Xiong, Jianhao Shen, Ye Yuan, Haiming Wang, Yichun Yin, Zhengying Liu, Lin Li, Zhijiang Guo, Qingxing Cao, Yinya Huang, Chuanyang Zheng, Xiaodan Liang, Ming Zhang, Qun Liu

STELLA: A Multimodal LLM for Protein Functional Annotation via Unified Sequence-Structure Encoding (2026.findings-acl)

Challenge:	Automated theorem proving (ATP) benchmarks focus on symbolic inference but rarely involve understanding complex number combination reasoning.
Approach:	They propose a benchmark that requires a model to reduce a trigonometric expression with step-by-step proof and evaluates a generative LM’s reasoning ability on formulas and ability to manipulate, group, and factor number terms.
Outcome:	The proposed benchmark evaluates a generative LM’s reasoning ability on formulas and ability to manipulate, group, and factor number terms.

Copied to clipboard

Hongwang Xiao, Wenjun Lin, Xi Chen, Hui Wang, Kai Chen, Jiashan Li, Yuancheng Sun, Sicheng Dai, Boya Wu, Qiwei Ye

UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Model (2023.findings-emnlp)

Challenge:	a multimodal protein language model (LLM) integrates sequence, structure, and function into functional annotation.
Approach:	They propose a multimodal protein language model that synergistically aligns bimodal representations with the textual modality to advance protein functional annotation.
Outcome:	The proposed model synergizes bimodal representations with the textual modality to advance protein functional annotation.

Copied to clipboard

Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Guohai Xu, Chenliang Li, Junfeng Tian, Qi Qian, Ji Zhang, Qin Jin, Liang He, Xin Lin, Fei Huang

ObfusLM: Privacy-preserving Language Model Service against Embedding Inversion Attacks (2025.acl-long)

Challenge:	Existing studies for visually-situated language understanding have shown shallow zero-shot visual text recognition ability when fed a low-resolution image with salient text information.
Approach:	They propose a model for universal OCR-free visually-situated language understanding based on the Multimodal Large Language Model (MLLM) their model is jointly finetuned on a wide range of visually situated language understanding tasks via a unified instruction format.
Outcome:	The proposed model achieves state-of-the-art ocr-free performance in 8 out of 10 visually-situated language understanding tasks across 5 domains: documents, tables, charts, natural images, and webpage screenshots.

Copied to clipboard

Yu Lin, Ruining Yang, Yunlong Mao, Qizhi Zhang, Jue Hong, Quanwei Cai, Ye Wu, Huiqi Liu, Zhiyu Chen, Bing Duan, Sheng Zhong

CMQCIC-Bench: A Chinese Benchmark for Evaluating Large Language Models in Medical Quality Control Indicator Calculation (2025.findings-acl)

Challenge:	Recent studies show that obfuscation techniques for MLaaS are susceptible to embedding inversion attacks (EIAs).
Approach:	They propose a model obfuscation framework that protects client inputs from embedding inversion attacks by obliviously obbing models.
Outcome:	The proposed framework outperforms existing works in utility by 10% with a nearly 80% resistance rate against embedding inversion attacks.

Copied to clipboard

Guangya Yu, Yanhao Li, Zongying Jiang, Yuxiong Jin, Li Dai, Yupian Lin, Ruihui Hou, Weiyan Zhang, Yongqi Fan, Qi Ye, Jingping Liu, Tong Ruan

Towards Quantifiable Dialogue Coherence Evaluation (2021.acl-long)

Challenge:	Medical quality control indicators are essential to assess the qualifications of healthcare institutions for medical services.
Approach:	They propose a Chinese electronic medical records-based dataset for MQCIC and propose CF-IR method that disentangles clinical fact verification and inferential rule reasoning actions.
Outcome:	The proposed method outperforms Chain-of-Thought methods on 20 representative LLMs, covering general and medical models.

Copied to clipboard

Zheng Ye, Liucun Lu, Lishan Huang, Liang Lin, Xiaodan Liang

Dialogue Systems & Conversational Agents

Challenge:	Existing automatic dialogue coherence evaluation metrics are expensive and high-latency, which cannot meet the requirements of a dialogue system.
Approach:	They propose a framework to train a quantifiable dialogue coherence metric that can reflect actual human rating standards.
Outcome:	Experimental results show that the model trained by QuantiDCE presents stronger correlations with human judgements than the other state-of-the-art metrics.

DetectBench: Can Large Language Model Detect and Piece Together Implicit Evidence? (2024.findings-emnlp)

Copied to clipboard

Zhouhong Gu, Lin Zhang, Xiaoxuan Zhu, Jiangjie Chen, Wenhao Huang, Yikai Zhang, Shusen Wang, Zheyu Ye, Yan Gao, Hongwei Feng, Yanghua Xiao

Towards Fair and Efficient De-identification: Quantifying the Efficiency and Generalizability of De-identification Approaches (2026.findings-eacl)

Challenge:	Existing LLMs' abilities to detect evidence in long contexts are far inferior to humans.
Approach:	They propose a benchmark to assess LLMs' abilities in evidence and multi-step commonsense reasoning within a long context.
Outcome:	The proposed method improves the performance of LLMs in evidence detection and commonsense reasoning.

Copied to clipboard

Noopur Zambare, Kiana Aghakasiri, Carissa Lin, Carrie Ye, J Ross Mitchell, Mohamed Abdalla

CausalMACE: Causality Empowered Multi-Agents in Minecraft Cooperative Tasks (2025.findings-emnlp)

Challenge:	a recent study has not examined their generalizability between formats, cultures, and genders.
Approach:	They evaluate large language models (LLMs) and small LLMs at clinical de-identification . they show that smaller models achieve comparable performance while substantially reducing inference cost .
Outcome:	The proposed models outperform larger models in de-identification tasks with limited data . the models can be fine-tuned with limited datasets to outperformed larger models .

Copied to clipboard

Qi Chai, Zhang Zheng, Junlong Ren, Deheng Ye, Zichuan Lin, Hao Wang

Challenge:	Existing work on multi-agent collaborative tasks in Minecraft is limited due to inefficiency and limited fault tolerance.
Approach:	They propose a framework that incorporates causality to manage dependencies among subtasks.
Outcome:	The proposed framework achieves state-of-the-art performance in multi-agent cooperative tasks of Minecraft.

Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Tasks (2026.findings-acl)

Copied to clipboard

Yuxiang Zhang, Jiangming Shu, Ye Ma, Xueyuan Lin, Shangxi Wu, Jitao Sang

MAPRO: Recasting Multi-Agent Prompt Optimization as Maximum a Posteriori Inference (2026.findings-eacl)

Challenge:	Existing approaches to managing working memory are based on external mechanisms that lack awareness of the agent’s reasoning state, leading to suboptimal decisions.
Approach:	They propose a framework that treats working memory management as learnable policy actions and enables joint optimization of information retention and task performance through end-to-end reinforcement learning.
Outcome:	The proposed framework matches models 16 larger while reducing average context length by 51%, with learned strategies that adapt to model capabilities and generalize across task complexities.

Copied to clipboard

Zheyuan Zhang, Lin Ge, Hongjiang Li, Weicheng Zhu, Chuxu Zhang, Yanfang Ye

CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP (2021.emnlp-main)

Challenge:	Large language models (LLMs) have demonstrated remarkable capabilities across diverse tasks.
Approach:	They propose a framework that optimizes MAS prompts as a maximum a posteriori problem and then iteratively updates agent prompts.
Outcome:	The proposed framework surpasses manual and automated benchmarks in multiple tasks and provides general guidelines for building more reliable and principled multi-agent systems in the future.

Copied to clipboard

Qinyuan Ye, Bill Yuchen Lin, Xiang Ren

Plug-and-Play Knowledge Injection for Pre-trained Language Models (2023.acl-long)

Challenge:	We study whether and how cross-task generalization ability can be acquired . we use CrossFit to standardize seen/unseen task partitions and evaluation protocols .
Approach:	They propose a problem setup for studying cross-task generalization ability which standardizes seen/unseen task partitions and data access during different learning stages.
Outcome:	The proposed model can be used to build few-shot learners across diverse tasks.

Copied to clipboard

Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Huadong Wang, Deming Ye, Chaojun Xiao, Xu Han, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

DT-Solver: Automated Theorem Proving with Dynamic-Tree Sampling Guided by Proof-level Value Function (2023.acl-long)

Challenge:	Existing knowledge injection methods are not suitable for enhancing pre-trained language models with external knowledge bases.
Approach:	They propose a plug-and-play knowledge injection method where knowledge bases are injected into frozen existing downstream models by a knowledge plugin.
Outcome:	The proposed method improves the performance of knowledge injection on knowledge-driven tasks while keeping model parameters frozen.

Copied to clipboard

Haiming Wang, Ye Yuan, Zhengying Liu, Jianhao Shen, Yichun Yin, Jing Xiong, Enze Xie, Han Shi, Yujun Li, Lin Li, Jian Yin, Zhenguo Li, Xiaodan Liang

Contrastive Token-Wise Meta-Learning for Unseen Performer Visual Temporal-Aligned Translation (2023.findings-acl)

Challenge:	Recent advances in neural theorem-proving resort to large language models and tree searches.
Approach:	They propose a Dynamic-Tree Driven Theorem Solver to accommodate general theoremes by guiding the search procedure with state confidence and proof-level values.
Outcome:	The proposed method outperforms state-of-the-art methods on two popular theorem-proving datasets with a 6.65% improvement on average in terms of success rate.

Copied to clipboard

Linjun Li, Tao Jin, Xize Cheng, Ye Wang, Wang Lin, Rongjie Huang, Zhou Zhao

Machine Translation

Challenge:	a novel generalization framework for visual temporal-aligned translation is proposed to transfer recognition skills to unseen performers . ambiguity in the visual sequence can hinder current methods for visual language translation .
Approach:	They propose a generalizable framework to transfer recognition skills to unseen performers . they use visual temporal-aligned translation to generate multiple words autoregressively .
Outcome:	The proposed framework is generalized to transfer recognition skills to unseen performers . it is compared with existing methods on lipreading and fingerspelling datasets .

A Simple but Effective Pluggable Entity Lookup Table for Pre-trained Language Models (2022.acl-short)

Copied to clipboard

Deming Ye, Yankai Lin, Peng Li, Maosong Sun, Zhiyuan Liu

Fast Quiet-STaR: Thinking Without Thought Tokens (2025.findings-emnlp)

Challenge:	Existing pre-trained language models cannot recall factual knowledge of entities exhibited in large-scale corpora, especially those rare entities.
Approach:	They propose to build a pluggable Entity Lookup Table (PELT) on demand by aggregating the entity’s output representations of multiple occurrences in the corpora.
Outcome:	The proposed model can transfer entity knowledge from out-of-domain corpora into PLMs with different architectures.

Copied to clipboard

Wei Huang, Yizhe Xiong, Xin Ye, Zhijie Deng, Hui Chen, Zijia Lin, Guiguang Ding

Rethinking the Multimodal Correlation of Multimodal Sequential Learning via Generalizable Attentional Results Alignment (2024.acl-long)

Challenge:	Large Language Models have achieved impressive performance across a range of tasks, but further gains require more than scaling up model sizes or training data.
Approach:	They propose a method that gradually reduces the number of thought tokens . this method allows models to internalize more abstract reasoning processes .
Outcome:	The proposed framework preserves the benefits of token-level reasoning while reducing computational cost.

Copied to clipboard

Tao Jin, Wang Lin, Ye Wang, Linjun Li, Xize Cheng, Zhou Zhao

PwnGPT: Automatic Exploit Generation Based on Large Language Models (2025.acl-long)

Challenge:	Existing studies have focused on the alignment of multimodal sequential learning using transformers.
Approach:	They propose a constrained scheme to align the multiple attentional results from both local and global perspectives.
Outcome:	The proposed scheme could align the multiple attentional results from both local and global perspectives, making the information capture more efficient.

Copied to clipboard

Wanzong Peng, Lin Ye, Xuetao Du, Hongli Zhang, Dongyang Zhan, Yunting Zhang, Yicheng Guo, Chen Zhang

Disentangling Reasoning Tokens and Boilerplate Tokens For Language Model Fine-tuning (2025.findings-acl)

Challenge:	Automated exploit generation (AEG) is the automatic discovery and exploitation of vulnerabilities against unknown targets.
Approach:	They propose an automatic exploit generation framework that automatically solves pwn challenges by using large language models.
Outcome:	The proposed framework improves the completion rate of exploits on the openAI o1-preview model and the GPT-4o model.

Copied to clipboard

Ziang Ye, Zhenru Zhang, Yang Zhang, Jianxin Ma, Junyang Lin, Fuli Feng

PCBERT: Parent and Child BERT for Chinese Few-shot NER (2022.coling-1)

Challenge:	Existing approaches to enhance agent capabilities for Large Language Models treat all tokens equally . however, reasoning tokens versus boilerplate tokens differ in importance and learning complexity . recent research has focused on enhancing agent capabilities in large language models .
Approach:	They propose a Shuffle-Aware Discriminator (SHAD) for adaptive token discrimination . they propose SHAD method which adaptively emphasizes reasoning tokens during fine-tuning .
Outcome:	The proposed method improves performance over standard fine-tuning methods.

Copied to clipboard

Peichao Lai, Feiyang Ye, Lin Zhang, Zhiwei Chen, Yanggeng Fu, Yingjie Wu, Yilei Wang

GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems (2020.emnlp-main)

Challenge:	Existing approaches to improve model performance on few-shot or zero-shot datasets are not effective for Chinese few- shot NER.
Approach:	They propose a prompt-based Parent and Child BERT for Chinese few-shot NER to train an annotating model on high-resource datasets and then discover more implicit labels on low-resourced datasets.
Outcome:	The proposed model can be used on Weibo and other Chinese NER datasets and it is shown to be effective in few-shot learning.

Copied to clipboard

Lishan Huang, Zheng Ye, Jinghui Qin, Liang Lin, Xiaodan Liang

Natural Language Interfaces

Challenge:	Existing evaluation metrics only consider surface features or utterance-level semantics, without explicitly considering the fine-grained topic transition dynamics of dialogue flows.
Approach:	They propose a graph-enhanced evaluation metric GRADE to evaluate dialogue coherence . GRADE incorporates utterance-level contextualized representations and fine-grained topic-level graph representations to improve communication logic.
Outcome:	The proposed evaluation metric outperforms state-of-the-art metrics on measuring diverse dialogue models in terms of Pearson and Spearman correlations with human judgments.

CURE: Critique-Driven Unified Reinforcement Learning for Test-Time Self-Improvement (2026.acl-long)

Copied to clipboard

Guirong Chen, Shuqi Ye, Wenkai Yang, Shiqi Shen, Guangyao Shen, Yankai Lin

Challenge:	Existing critique-guided methods fail to equip models with the autonomous improvement capabilities required for test-time scaling.
Approach:	They propose a framework that jointly optimizes a single policy for standard solving, critiquing, and guided re-exploration.
Outcome:	The proposed framework maintains competitive single-turn performance and unlocks effective inference-time scaling.

The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models (2025.naacl-long)

Copied to clipboard

Seungone Kim, Juyoung Suk, Ji Yong Cho, Shayne Longpre, Chaeeun Kim, Dongkeun Yoon, Guijin Son, Yejin Cho, Sheikh Shafayat, Jinheon Baek, Sue Hyun Park, Hyeonbin Hwang, Jinkyung Jo, Hyowon Cho, Haebin Shin, Seongyun Lee, Hanseok Oh, Noah Lee, Namgyu Ho, Se June Joo, Miyoung Ko, Yoonjoo Lee, Hyungjoo Chae, Jamin Shin, Joel Jang, Seonghyeon Ye, Bill Yuchen Lin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, Minjoon Seo

Enhancing Chain-of-Thought Reasoning with Critical Representation Fine-tuning (2025.acl-long)

Challenge:	a recent study evaluated language models using abstract evaluation criteria that lack the flexibility and granularity of human assessment.
Approach:	They propose a benchmark to evaluate nine distinct language models' capabilities . they use instance-specific evaluation criteria to mirror human evaluation .
Outcome:	The proposed benchmark evaluates nine distinct capabilities of language models across 77 tasks.

Copied to clipboard

Chenxi Huang, Shaotian Yan, Liang Xie, Binbin Lin, Sinan Fan, Yue Xin, Deng Cai, Chen Shen, Jieping Ye

Contextual Domain Classification with Temporal Representations (2021.naacl-industry)

Challenge:	Representation Fine-tuning (ReFT) is a proposed method for improving parameter efficiency . however, it yields suboptimal performance, as fixed-position representations have uncertain impact on outputs .
Approach:	They propose a method that fine-tunes critical representations in a low-rank linear subspace while freezing the base model.
Outcome:	The proposed method improves accuracy of LLaMA-2-7B and ReFT by 18.2 and 3.8 on GSM8K.

Copied to clipboard

Tzu-Hsiang Lin, Yipeng Shi, Chentao Ye, Yang Fan, Weitong Ruan, Emre Barut, Wael Hamza, Chengwei Su

Information Retrieval

Challenge:	Existing studies that incorporate context in SLU have focused on domains where context is limited to a few minutes.
Approach:	They propose temporal representations that combine wall-clock second difference and turn order offset information to utilize both recent and distant context in a novel large-scale setup.
Outcome:	The proposed model reduces 13.04% of classification errors compared to baseline . previous studies have focused on domains where context is limited to a few minutes .

DebugBench: Evaluating Debugging Capability of Large Language Models (2024.findings-acl)

Copied to clipboard

Runchu Tian, Yining Ye, Yujia Qin, Xin Cong, Yankai Lin, Yinxu Pan, Yesai Wu, Hui Haotian, Liu Weichuan, Zhiyuan Liu, Maosong Sun

An Inversion Attack Against Obfuscated Embedding Matrix in Language Model Inference (2024.emnlp-main)

Challenge:	Large language models (LLMs) have demonstrated exceptional coding capabilities, but their debugging capabilities remain relatively unexplored.
Approach:	They propose a debugging benchmark consisting of 4,253 LLMs with four major bug categories and 18 minor types in C++, Java, and Python.
Outcome:	The proposed benchmark covers four major bug categories and 18 minor types in C++, Java, and Python.

Copied to clipboard

Yu Lin, Qizhi Zhang, Quanwei Cai, Jue Hong, Wu Ye, Huiqi Liu, Bing Duan

A Scalable Multi-LLM Collaboration System with Retrieval-based Selection and Exploration-Exploitation-Driven Enhancement (2026.acl-long)

Challenge:	Recent studies have explored transforming user inputs to obfuscated embedded vectors, so that the data will not be eavesdropped by service providers.
Approach:	They propose to transform user inputs to obfuscated embedded vectors so that the data will not be eavesdropped by service providers.
Outcome:	The proposed inversion attack can recover user input 100% from the obfuscated vectors without a solid and deliberate security design and analysis .

Copied to clipboard

Shengji Tang, Jianjian Cao, Weihao Lin, Jiale Hong, Bo Zhang, Shuyue Hu, Lei Bai, Tao Chen, Wanli Ouyang, Peng Ye

MNER-MI: A Multi-image Dataset for Multimodal Named Entity Recognition in Social Media (2024.lrec-main)

Challenge:	Existing multi-LLM collaboration systems often encounter scalability challenges when integrating new LLMs and tasks.
Approach:	They propose a Scalable Multi-LLM Collaboration System to coordinate multiple open-source LLMs.
Outcome:	The proposed system outperforms prevailing closed-source LLMs on eight mainstream benchmarks on multiple tasks.

Copied to clipboard

Shizhou Huang, Bo Xu, Changqun Li, Jiabo Ye, Xin Lin

ScratchEval: Are GPT-4o Smarter than My Child? Evaluating Large Multimodal Models with Visual Programming Challenges (2025.naacl-short)

Challenge:	Recent research has focused on multimodal named entity recognition (MNER) but current approaches focus on text and a single accompanying image, leaving a significant research gap in multi-image scenarios.
Approach:	They propose to construct a human-annotated MNER dataset with multiple images called MNER-MI and a temporal prompt model with multiple image to address the new challenges in multi-image scenarios.
Outcome:	The proposed method achieves state-of-the-art results on both MNER-MI and MNER -MI-Plus, demonstrating its effectiveness.

Copied to clipboard

Rao Fu, Ziyang Luo, Hongzhan Lin, Zhen Ye, Jing Ma

ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Use (2025.acl-long)

Challenge:	Recent advances in large multimodal models (LMMs) have demonstrated impressive code generation capabilities, primarily evaluated through image-to-code benchmarks.
Approach:	They propose a visual programming reasoning benchmark based on Scratch, a block-based visual programming language widely used in children’s programming education.
Outcome:	The proposed framework evaluates the visual programming ability of large multimodal models by integrating visual elements and embedded programming logic.

Copied to clipboard

Junjie Ye, Zhengyin Du, Xuesong Yao, Weijian Lin, Yufei Xu, Zehui Chen, Zaiyuan Wang, Sining Zhu, Zhiheng Xi, Siyu Yuan, Tao Gui, Qi Zhang, Xuanjing Huang, Jiecao Chen

Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub (2025.acl-long)

Challenge:	Effective evaluation of multi-hop tool use is critical for analyzing the understanding, reasoning, and function-calling capabilities of large language models.
Approach:	They propose a dataset that provides rigorous evaluation of multi-hop tool use.
Outcome:	The proposed model achieves 49.04% accuracy across five model families.

Copied to clipboard

Bohan Lyu, Xin Cong, Heyang Yu, Pan Yang, Cheng Qian, Zihe Wang, Yujia Qin, Yining Ye, Yaxi Lu, Chen Qian, Zhong Zhang, Yukun Yan, Yankai Lin, Zhiyuan Liu, Maosong Sun