Papers by Peng Lin

Self-supervised Quantized Representation for Seamlessly Integrating Knowledge Graphs with Large Language Models (2025.acl-long)

Challenge:	Experimental results show that pre-trained language model GPT2 can generate better continuations by learning to generate the in the fine-tuning stage.
Approach:	They conduct experiments on an English essay dataset using Chinese-GPT2 . they find that the model can generate better continuations by learning to generate the in the fine-tuning stage.
Outcome:	The pre-trained language model GPT2 can generate better continuations by learning to generate the in the fine-tuning stage.

Copied to clipboard

Qika Lin, Tianzhe Zhao, Kai He, Zhen Peng, Fangzhi Xu, Ling Huang, Jingying Ma, Mengling Feng

Farewell Freebase: Migrating the SimpleQuestions Dataset to DBpedia (C18-1)

Challenge:	Large Language Models (LLMs) are gaining popularity due to their lack of knowledge hallucination and lack of a coherent model.
Approach:	They propose a self-supervised quantized representation method to compress KG structural and semantic knowledge into discrete codes that align the format of language sentences.
Outcome:	The proposed framework outperforms existing unsupervised methods producing more distinguishable codes on KG link prediction and triple classification tasks.

Copied to clipboard

Michael Azmy, Peng Shi, Jimmy Lin, Ihab Ilyas

Summarising Historical Text in Modern Languages (2021.eacl-main)

Challenge:	Existing datasets for question answering over knowledge graphs lack answer triples from Freebase . a defunct knowledge graph makes it difficult to build "real-world" question answering systems .
Approach:	They propose a benchmark dataset for simple question answering over knowledge graphs that maps SimpleQuestions entities and predicates from Freebase to DBpedia.
Outcome:	The proposed dataset provides simple yet strong baselines with and without neural networks.

Copied to clipboard

Xutan Peng, Yi Zheng, Chenghua Lin, Advaith Siddharthan

Text Generation

Challenge:	Historical text summarisation is a routine for historians and digital humanities researchers but has never been automated.
Approach:	They propose a model that can be trained even with no cross-lingual data and further benchmark it against state-of-the-art algorithms.
Outcome:	The proposed model outperforms standard cross-lingual benchmarks on historical text summarisation task and identifies distinctness and value of the dataset.

RoadMapper: A Multi-Agent System for Roadmap Generation of Solving Complex Research Problems (2026.findings-acl)

Copied to clipboard

Jiacheng Liu, Zichen Tang, Zhongjun Yang, Xinyi Hu, Xueyuan Lin, Linwei Jia, Ruofei Bai, Rongjin Li, Shiyao Peng, Haocheng Gao, Haihong E

RWKV: Reinventing RNNs for the Transformer Era (2023.findings-emnlp)

Challenge:	Existing tools to generate structured content for research tasks are limited in their ability to generate high-quality roadmaps.
Approach:	They propose a benchmark to evaluate the ability of large language models (LLMs) to generate high-quality roadmaps for solving complex research problems.
Outcome:	The proposed system can improve LLMs’ ability for roadmap generation while saving 84% of the time required by human experts.

Copied to clipboard

Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella, Kranthi Gv, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartłomiej Koptyra, Hayden Lau, Jiaju Lin, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Johan Wind, Stanisław Woźniak, Zhenyuan Zhang, Qinghua Zhou, Jian Zhu, Rui-Jie Zhu

FOLIO: Natural Language Reasoning with First-Order Logic (2024.emnlp-main)

Challenge:	recurrent neural networks struggle to match the performance of Transformers due to limitations in parallelization and scalability.
Approach:	They propose a model architecture that combines the efficient parallelizable training of transformers with the efficient inference of RNNs.
Outcome:	The proposed model performs on par with similarly sized RNNs, suggesting future work can leverage this architecture to create more efficient models.

Copied to clipboard

Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, Lucy Sun, Alexander Wardle-Solano, Hannah Szabó, Ekaterina Zubova, Matthew Burtell, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kasai, Tao Yu, Rui Zhang, Alexander Fabbri, Wojciech Kryscinski, Semih Yavuz, Ye Liu, Xi Lin, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Rex Ying, Arman Cohan, Dragomir Radev

Reasoning

Challenge:	Existing benchmarks for logical reasoning in large language models lack language naturalness or limited complexity.
Approach:	They propose to use first-order logic annotations to evaluate logical reasoning capabilities of large language models.
Outcome:	The proposed dataset evaluates the FOL reasoning ability of supervised fine-tuning on medium-sized language models.

Mitigating Bias for Question Answering Models by Tracking Bias Influence (2024.naacl-long)

Copied to clipboard

Mingyu Ma, Jiun-Yu Kao, Arpit Gupta, Yu-Hsiang Lin, Wenbo Zhao, Tagyoung Chung, Wei Wang, Kai-Wei Chang, Nanyun Peng

Question Answering

Challenge:	Existing literature observes bias in question answering (QA) models, but there is no method to mitigate it.
Approach:	They propose an approach to mitigate the bias of question answering models by observing the influence of a query instance on another instance.
Outcome:	The proposed method reduces bias level in all 9 bias categories while maintaining comparable QA accuracy.

Adaptive Hyper-parameter Learning for Deep Semantic Retrieval (2023.emnlp-industry)

Copied to clipboard

Mingming Li, Chunyuan Yuan, Huimu Wang, Peng Wang, Jingwei Zhuo, Binbin Wang, Lin Liu, Sulong Xu

Information Retrieval

Challenge:	Existing methods for deep semantic retrieval are highly sensitive to hyper-parameters . a novel adaptive metric learning method is proposed to overcome this limitation .
Approach:	They propose a method that adaptively obtains hyper-parameters without fixed or extra-trainable hyper-parmeters . they adopt a symmetric metric learning method to mitigate model collapse issues .
Outcome:	The proposed method outperforms existing methods on a real-world dataset and brings economic benefits.

MAKAR: a Multi-Agent framework based Knowledge-Augmented Reasoning for Grounded Multimodal Named Entity Recognition (2025.emnlp-main)

Copied to clipboard

Xinkui Lin, Yuhui Zhang, Yongxiu Xu, Kun Huang, Hongzhang Mu, Yubin Wang, Gaopeng Gou, Li Qian, Li Peng, Wei Liu, Jian Luan, Hongbo Xu

A Semi-supervised Scalable Unified Framework for E-commerce Query Classification (2025.acl-industry)

Challenge:	Existing methods for GMNER fail to address semantic ambiguity caused by polysemy and long-tail distribution of datasets.
Approach:	They propose a framework for Grounded Multimodal Named Entity Recognition that leverages a Multimodal Large Language Model to address semantic ambiguity.
Outcome:	Extensive experiments show that the proposed framework outperforms existing methods on two benchmark datasets.

Copied to clipboard

Chunyuan Yuan, Chong Zhang, Zhen Fang, Ming Pang, Xue Jiang, Changping Peng, Zhangang Lin, Ching Law

Information Retrieval

Challenge:	Existing query classification methods rely on posterior click behavior to construct training samples, resulting in insufficient prior information for modeling.
Approach:	They propose a semi-supervised scaleable unified framework that integrates enhanced modules to unify query classification tasks.
Outcome:	The proposed framework outperforms the state-of-the-art models in offline and online A/B experiments.

Coreferential Reasoning Learning for Language Representation (2020.emnlp-main)

Copied to clipboard

Deming Ye, Yankai Lin, Jiaju Du, Zhenghao Liu, Peng Li, Maosong Sun, Zhiyuan Liu

Packed Levitated Marker for Entity and Relation Extraction (2022.acl-long)

Challenge:	Existing language representation models cannot explicitly handle coreference, which is essential to the coherent understanding of the whole discourse.
Approach:	They propose a language representation model that captures coreferential relations in context.
Outcome:	The proposed model can achieve significant improvements on downstream NLP tasks while maintaining comparable performance to baseline models on other common NLP task.

Copied to clipboard

Deming Ye, Yankai Lin, Peng Li, Maosong Sun

Knowledge Inheritance for Pre-trained Language Models (2022.naacl-main)

Challenge:	Existing work on entity and relation extraction ignores the interrelation between spans . a novel approach to extract better span representations from pre-trained languages is needed .
Approach:	They propose a span representation approach that packs Levitated Markers to consider interrelation between spans.
Outcome:	The proposed model improves on baselines on six NER benchmarks and achieves a 4.1%-4.3% strict relation F1 improvement with higher speed over previous state-of-the-art models.

Copied to clipboard

Yujia Qin, Yankai Lin, Jing Yi, Jiajie Zhang, Xu Han, Zhengyan Zhang, Yusheng Su, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

MAVEN: A Massive General Domain Event Detection Dataset (2020.emnlp-main)

Challenge:	Existing large-scale pre-trained language models are mainly trained from scratch individually, ignoring that many well-taught PLMs are available.
Approach:	They propose a pre-training framework called knowledge inheritance and propose auxiliary supervision to efficiently learn larger PLMs.
Outcome:	The proposed framework can be used to train large-scale language models with huge parameters and a large dataset can be adapted to domain adaptation and knowledge transfer.

Copied to clipboard

Xiaozhi Wang, Ziqi Wang, Xu Han, Wangyi Jiang, Rong Han, Zhiyuan Liu, Juanzi Li, Peng Li, Yankai Lin, Jie Zhou

MAVEN-ARG: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation (2024.acl-long)

Challenge:	Existing datasets exhibit data scarcity and limited coverage of general-domain events.
Approach:	They present a MAssive eVENt detection dataset which contains 4,480 Wikipedia documents and 168 event types.
Outcome:	The proposed dataset shows that existing methods cannot achieve promising results on the small datasets.

Copied to clipboard

Xiaozhi Wang, Hao Peng, Yong Guan, Kaisheng Zeng, Jianhui Chen, Lei Hou, Xu Han, Yankai Lin, Zhiyuan Liu, Ruobing Xie, Jie Zhou, Juanzi Li

Beyond Facts- Benchmarking Distributional Reading Comprehension in Large Language Models (2026.findings-acl)

Challenge:	Existing datasets for event understanding have limited coverage due to complexity of tasks.
Approach:	They propose a dataset that augments MAVEN datasets with event argument annotations . they propose 98,591 events and 290,613 arguments obtained with laborious human annotation .
Outcome:	The proposed dataset is the first all-in-one dataset supporting event detection, event argument extraction, and event relation extraction.

Copied to clipboard

Pei-Fu Guo, Ya An Tsai, Chun-Chia Hsu, Kai-Xin Chen, Yun-Da Tsai, Kai-Wei Chang, Nanyun Peng, Mi-Yen Yeh, Shou-De Lin

LiveCLKTBench: Towards Reliable Evaluation of Cross-Lingual Knowledge Transfer in Multilingual LLMs (2026.acl-long)

Challenge:	Existing reading comprehension benchmarks focus on factual information, but many real-world tasks require distributional knowledge expressed across text.
Approach:	They propose a reading comprehension benchmark for LLMs to evaluate their ability to infer distributional knowledge from natural language.
Outcome:	Experiments with multiple LLMs show that the model outperforms baselines, but performance varies widely across distribution types and characteristics.

Copied to clipboard

Pei-Fu Guo, Yun-Da Tsai, Chun-Chia Hsu, Kai-Xin Chen, Ya An Tsai, Kai-Wei Chang, Nanyun Peng, Mi-Yen Yeh, Shou-De Lin

More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction (2020.aacl-main)

Challenge:	Evaluating cross-lingual knowledge transfer in large language models is challenging, as correct answers in a target language may arise either from genuine transfer or from prior exposure during pre-training.
Approach:	They propose a pipeline to isolate and measure cross-lingual knowledge transfer by identifying self-contained, time-sensitive knowledge entities from real-world domains and generating factual questions.
Outcome:	The proposed pipeline analyzes multiple LLMs across five languages and shows that cross-lingual transfer is strongly influenced by linguistic distance and often asymmetric across language directions.

Copied to clipboard

Xu Han, Tianyu Gao, Yankai Lin, Hao Peng, Yaoliang Yang, Chaojun Xiao, Zhiyuan Liu, Peng Li, Jie Zhou, Maosong Sun

Intrinsic Mutual Information as a Modulator for Preference Optimization (2026.findings-acl)

Challenge:	Existing methods for extracting relational facts from text have been successful . but with explosion of Web text, human knowledge is increasing drastically .
Approach:	They propose to improve relation extraction methods to extract relational facts from text . they analyze existing methods and show promising directions towards more powerful RE .
Outcome:	The proposed methods can extract relational facts from text, but they are still lacking in the current field.

Copied to clipboard

Peng Liao, Peijia Zheng, Lingbo Li, Shangsong Liang, Lin Chen

Challenge:	Existing methods for offline preference optimization involve additional hyperparameter tuning, resulting in substantial time overhead.
Approach:	They propose a lightweight framework for offline preference optimization that leverages hyperparameter modulation to decouple preference contributions.
Outcome:	The proposed framework achieves superior performance over existing methods while reducing training overhead by more than 15%.

Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning (2024.findings-acl)

Copied to clipboard

Naibin Gu, Peng Fu, Xiyu Liu, Bowen Shen, Zheng Lin, Weiping Wang

LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints (2024.findings-emnlp)

Challenge:	Existing methods for fine-tuning large language models are inefficient and redundant . a light-PEFT framework can be used to prune redundant parameters during training .
Approach:	They propose a parameter-efficient fine-tuning framework that freezes most parameters of the foundation model and finetuns only a small number of parameters.
Outcome:	The proposed framework achieves training and inference speedup, reduces memory usage, and maintains comparable performance and plug-and-play feature of PEFT.

Copied to clipboard

Thomas Palmeira Ferraz, Kartik Mehta, Yu-Hsiang Lin, Haw-Shiuan Chang, Shereen Oraby, Sijia Liu, Vivek Subramanian, Tagyoung Chung, Mohit Bansal, Nanyun Peng

Bridging the Gap between Relevance Matching and Semantic Matching for Short Text Similarity Modeling (D19-1)

Challenge:	Recent studies have shown that LLMs struggle with instructions containing multiple constraints.
Approach:	They propose a self-correction pipeline that decomposes the original instruction into a list of constraints and uses a Critic model to decide when and where the LLM’s response needs refinement.
Outcome:	The proposed model outperforms GPT-4 on RealInstruct and IFEval even with weak feedback.

Copied to clipboard

Jinfeng Rao, Linqing Liu, Yi Tay, Wei Yang, Peng Shi, Jimmy Lin

VDebugger: Harnessing Execution Feedback for Debugging Visual Programs (2024.findings-emnlp)

Challenge:	Existing techniques for relevance and semantic matching cannot be easily adapted to the other.
Approach:	They propose a model that incorporates a hybrid encoder module, a relevance matching module and co-attention mechanisms that capture context-aware semantic relatedness.
Outcome:	The proposed model incorporates a hybrid encoder module, a relevance matching module and co-attention mechanisms that capture context-aware semantic relatedness.

Copied to clipboard

Xueqing Wu, Zongyu Lin, Songyan Zhao, Te-Lin Wu, Pan Lu, Nanyun Peng, Kai-Wei Chang

MoEL: Mixture of Empathetic Listeners (D19-1)

Challenge:	Visual programs are executable code generated by large language models to address visual reasoning problems.
Approach:	They propose a critic-refiner framework that localizes and debugs visual programs by tracking execution step by step.
Outcome:	The proposed framework detects and corrects program errors leveraging detailed execution feedback, improving interpretability and accuracy.

Copied to clipboard

Zhaojiang Lin, Andrea Madotto, Jamin Shin, Peng Xu, Pascale Fung

Parameter Importance is Not Static: Evolving Parameter Isolation for Supervised Fine-Tuning (2026.acl-long)

Challenge:	Neural network approaches for conversation models have shown to be successful in generating fluent and relevant responses.
Approach:	They propose a novel end-to-end approach for modeling empathy in dialogue systems by using Mixture of Empathetic Listeners (MoEL).
Outcome:	The proposed model outperforms multitask training baseline in terms of empathy, relevance, and fluency.

Copied to clipboard

Zekai Lin, Chao Xue, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Lei Jiang, Yu Lu, Bob Simons, Shuang Liang, Minlong Peng

CTRLEval: An Unsupervised Reference-Free Metric for Evaluating Controlled Text Generation (2022.acl-long)

Challenge:	Recent approaches to fine-tuning of large language models suffer from task interference and catastrophic forgetting.
Approach:	They propose a fine-tuning framework that adapts isolation decisions based on online estimates of parameter importance.
Outcome:	The proposed framework reduces interference and forgetting while releasing outdated parameters to recover plasticity.

Copied to clipboard

Pei Ke, Hao Zhou, Yankai Lin, Peng Li, Jie Zhou, Xiaoyan Zhu, Minlie Huang

Low-Resource NLP

Challenge:	Existing reference-free metrics have obvious limitations for evaluating controlled text generation models.
Approach:	They propose an unsupervised reference-free metric which evaluates controlled text generation from different aspects by formulating each aspect into multiple text infilling tasks.
Outcome:	The proposed metric has higher correlations with human judgments while obtaining better generalization of evaluating generated texts from different models and with different qualities.

Cross-Lingual Training of Neural Models for Document Ranking (2020.findings-emnlp)

Copied to clipboard

Peng Shi, He Bai, Jimmy Lin

RepoGenesis: Benchmarking End-to-End Microservice Generation from Readme to Repository (2026.acl-long)

Challenge:	a recent study shows that multi-lingual BERT models can be used for document ranking in non-English languages . a blog post by Google suggests that the company is exploring this approach to improve web search across a number of languages.
Approach:	They propose to leverage relevance judgments in English to train neural document ranking models for mono-lingual retrieval in multiple target languages.
Outcome:	The proposed approach improves search quality in non-English languages while requiring low resources.

Copied to clipboard

Zhiyuan Peng, Xin Yin, Pu Zhao, Fangkai Yang, Lu Wang, Ran Jia, Xu Chen, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang

Challenge:	Existing benchmarks focus on isolated function/class-level generation, neglecting complete microservice repository generation.
Approach:	They propose a multilingual benchmark for repository-level end-to-end web microservice generation that reflects real-world development workflows.
Outcome:	The benchmark compared 106 repositories across 18 domains and 11 frameworks and 1,258 API endpoints and 2,335 test cases.

Cross-lingual Text-to-SQL Semantic Parsing with Representation Mixup (2022.findings-emnlp)

Copied to clipboard

Peng Shi, Linfeng Song, Lifeng Jin, Haitao Mi, He Bai, Jimmy Lin, Dong Yu

MT-Video-Bench: A Holistic Video Understanding Benchmark for Evaluating Multimodal LLMs in Multi-Turn Dialogues (2026.findings-acl)

Challenge:	Experimental results show that Rex can benefit from cross-lingual training and improve the effectiveness of semantic parsers.
Approach:	They propose a Representation Mixup Framework for effectively exploiting translations in the cross-lingual Text-to-SQL task.
Outcome:	The proposed framework can benefit from cross-lingual training and improve the effectiveness of semantic parsers, achieving state-of-the-art performance.

Copied to clipboard

Yaning Pan, Qianqian Xie, Guohui Zhang, Zekun Moore Wang, Yongqian Wen, Yuanxing Zhang, Haoxuan Hu, Zhiyu Pan, Yibing Huang, Zhidong Gan, Yonghong Lin, An Ping, Shihao Li, Yanghai Wang, Tianhao Peng, Jiaheng Liu

FLAG-TRADER: Fusion LLM-Agent with Gradient-based Reinforcement Learning for Financial Trading (2025.findings-acl)

Challenge:	Existing evaluation benchmarks for Multimodal Large Language Models (MLLMs) focus on single-turn question answering, overlooking the complexity of multi-turn dialogues in real-world scenarios.
Approach:	They propose a video understanding benchmark for MLLMs in multi-turn dialogues that assesses six core competencies that focus on perceptivity and interactivity.
Outcome:	The MT-Video-Bench evaluates 1,000 multi-turn dialogues from diverse domains and reveals significant performance discrepancies and limitations in handling multi-turned video dialogues.

Copied to clipboard

Guojun Xiong, Zhiyang Deng, Keyi Wang, Yupeng Cao, Haohang Li, Yangyang Yu, Xueqing Peng, Mingquan Lin, Kaleb E Smith, Xiao-Yang Liu, Jimin Huang, Sophia Ananiadou, Qianqian Xie

Video-LLaVA: Learning United Visual Representation by Alignment Before Projection (2024.emnlp-main)

Challenge:	Large language models (LLMs) have impressive reasoning capabilities in financial tasks, but struggle with multi-step, goal-oriented scenarios in interactive financial markets.
Approach:	They propose a framework that integrates large language models with gradient-driven reinforcement learning (RL) policy optimization.
Outcome:	The proposed framework improves performance in trading and other financial domain tasks.

Copied to clipboard

Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, Li Yuan

HammerBench: Fine-Grained Function-Calling Evaluation in Real Mobile Assistant Scenarios (2025.findings-acl)

Challenge:	Existing approaches to visual-language understanding lack unified tokenization for images and videos . lack of unified visual representations makes it difficult to learn multi-modal interactions from poor projection layers.
Approach:	They propose to unify visual representation into the language feature space to advance the foundational LLM towards a unified LVLM.
Outcome:	The proposed model outperforms Video-ChatGPT on image benchmarks and on 9 image benchmark benchmarks.

Copied to clipboard

Jun Wang, Jiamu Zhou, Xihuai Wang, Xiaoyun Mo, Haoyu Zhang, Qiqiang Lin, Jincheng Jincheng, Muning Wen, Weinan Zhang, Qiuying Peng, Jun Wang

ToolPRM: Fine-Grained Inference Scaling of Structured Outputs for Function Calling (2026.acl-long)

Challenge:	Evaluating the performance of LLMs in multi-turn interactions presents significant challenges due to the complexity and variability of user behavior.
Approach:	They propose a benchmark framework for assessing LLMs’ function-calling capabilities in multi-turn dialogues.
Outcome:	The proposed framework is based on a dataset derived from popular mobile apps and anonymized user logs.

Copied to clipboard

Jianghao Lin, Yuanyuan Shi, Xin Peng, Renjie Ding, Hairui Wang, Yuxuan Peng, Bizhe Bai, Weixi Song, Fengshuo Bai, Huacan Chai, Weinan Zhang, Fei Huang, Ying Wen

Challenge:	Existing research on inference scaling focuses on unstructured output generation tasks, such as mathematical problems.
Approach:	They propose an inference-scaling framework that combines fine-grained beam search with ToolPRM, a process reward model scoring each intra-call decision.
Outcome:	The proposed framework outperforms outcome and coarse-grained reward models in predictive accuracy and yields consistent test-time gains on multiple function-calling benchmarks.

Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis (2021.naacl-main)

Copied to clipboard

Xutan Peng, Guanyi Chen, Chenghua Lin, Mark Stevenson

Improve Interpretability of Neural Networks via Sparse Contrastive Coding (2022.findings-emnlp)

Challenge:	Knowledge Graph Embeddings (KGEs) have been explored in recent years due to their promise for a wide range of applications.
Approach:	They propose a KGE framework which can reduce the training time and carbon footprint by orders of magnitudes compared with state-of-the-art approaches.
Outcome:	The proposed framework reduces the training time and carbon footprint by orders of magnitudes compared with state-of-the-art approaches while producing competitive performance.

Copied to clipboard

Junhong Liu, Yijie Lin, Liang Jiang, Jia Liu, Zujie Wen, Xi Peng

Explainability & Interpretability in NLP

Challenge:	XAI has achieved remarkable advances, but few efforts have been devoted to solving the problem.
Approach:	They propose a model-agnostic explanation method termed Sparse Contrastive Coding . they use model-based explanations to explain the black-box in a more model-oriented way .
Outcome:	The proposed method outperforms five state-of-the-art methods in interpretability and classification metrics.

ORANGE: Text-video Retrieval via Watch-time-aware Heterogeneous Graph Contrastive Learning (2023.emnlp-industry)

Copied to clipboard

Yucheng Lin, Tim Chang, Yaning Chang, Jianqiang Ma, Donghui Li, Ting Peng, Zang Li, Zhiyi Zhou, Feng Wang

Disentangle-based Continual Graph Representation Learning (2020.emnlp-main)

Challenge:	Existing methods for text-video retrieval focus on informative representations and delicate matching mechanisms, but real-world scenarios often involve brief, ambiguous queries and low-quality videos.
Approach:	They propose a novel method to learn informative embeddings for queries and videos . they use a watch-time-aware contrastive learning paradigm to capture dependencies .
Outcome:	The proposed method is effective in a real-world video-search service.

Copied to clipboard

Xiaoyu Kou, Yankai Lin, Shaobo Liu, Peng Li, Jie Zhou, Yan Zhang

Past, Present, and Future: Conversational Emotion Recognition through Structural Modeling of Psychological Knowledge (2021.findings-emnlp)

Challenge:	Existing graph embedding methods overlook streaming nature of incoming data in real-world applications.
Approach:	They propose a disentangle-based continual graph representation learning framework inspired by the human’s ability to learn procedural knowledge.
Outcome:	The proposed framework outperforms state-of-the-art continual graph representation learning framework and alleviate catastrophic forgetting problem.

Copied to clipboard

Jiangnan Li, Zheng Lin, Peng Fu, Weiping Wang

CLEVE: Contrastive Pre-training for Event Extraction (2021.acl-long)

Challenge:	Conversational Emotion Recognition (CER) is a task to predict the emotion of an utterance in the context of a conversation.
Approach:	They propose a pSychological-Knowledge-Aware Interaction Graph to model the emotional state of an utterance in the context of a conversation.
Outcome:	The proposed method achieves state-of-the-art and competitive performance on four popular CER datasets.

Copied to clipboard

Ziqi Wang, Xiaozhi Wang, Xu Han, Yankai Lin, Lei Hou, Zhiyuan Liu, Peng Li, Juanzi Li, Jie Zhou

Getting To Know You: User Attribute Extraction from Dialogues (2020.lrec-1)

Challenge:	Existing EE methods do not model event characteristics from large unsupervised data.
Approach:	They propose a contrastive pre-training framework for event extraction to better learn event knowledge from large unsupervised data and their semantic structures.
Outcome:	The proposed framework improves on ACE 2005 and MAVEN datasets on event extraction tasks.

Copied to clipboard

Chien-Sheng Wu, Andrea Madotto, Zhaojiang Lin, Peng Xu, Pascale Fung

Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defense (2025.naacl-long)

Challenge:	a new method to extract user attributes from dialogues is needed to improve user understanding.
Approach:	They propose to leverage dialogues with conversational agents to automatically extract user attributes from dialogues.
Outcome:	The proposed model surpasses retrieval and generation baselines on human evaluation.

Copied to clipboard

Yang Ouyang, Hengrui Gu, Shuhang Lin, Wenyue Hua, Jie Peng, Bhavya Kailkhura, Meijun Gao, Tianlong Chen, Kaixiong Zhou

MEDSYN: Benchmarking Multi-EviDence SYNthesis in Complex Clinical Cases for Multimodal Large Language Models (2026.findings-acl)

Challenge:	Existing methods to defend against jailbreak attacks exploit vulnerabilities to elicit unintended or harmful outputs.
Approach:	They propose a method to defend against jailbreak attacks by patching specific layers within large language models through self-augmented datasets.
Outcome:	The proposed approach reduces harmfulness and attack success rate of jailbreak attacks without compromising utility for benign queries compared to previous methods.

Copied to clipboard

Boqi Chen, Xudong Liu, Jiachuan Peng, Marianne Frey-Marti, Kyle Lam, Bang Zheng, Lin Li, Jianing Qiu

OpenS2S: Advancing Fully Open-Source End-to-End Empathetic Large Speech Language Model (2025.emnlp-demos)

Challenge:	Existing benchmarks for multimodal large language models do not capture real-world clinical complexity.
Approach:	They evaluate multilingual, multimodal multimodal models of clinical cases with up to 7 distinct visual clinical evidence types per case.
Outcome:	The proposed model outperforms human models on differential diagnosis (DDx) generation and final diagnosis (FDx) selection.

Copied to clipboard

Chen Wang, Tianyu Peng, Wen Yang, YiNan Bai, Guangfu Wang, Jun Lin, Lanpeng Jia, Lingxiang Wu, Jinqiao Wang, Chengqing Zong, Jiajun Zhang

HierDiffuse: Progressive Diffusion for Robust Interest Fusion in CTR Prediction (2025.emnlp-industry)

Challenge:	Empathetic speech models are increasingly closed off, leaving details about the architecture, data and development opaque to researchers.
Approach:	They propose an open-source empathetic speech-to-text model with a streaming interleaved decoding architecture and a data pipeline to enable end-to end training.
Outcome:	The proposed model is open-source and transparent, with no data or data required to build it.

Copied to clipboard

Ziheng Ni, Congcong Liu, Yuying Chen, Zhiwei Fang, Changping Peng, Zhangang Lin, Ching Law, Jingping Shao

ECom-Bench: Can LLM Agent Resolve Real-World E-commerce Customer Support Issues? (2025.emnlp-industry)

Challenge:	Existing approaches fuse long-term behavioral profiles and short-term interactions, suffering from representational misalignment and noise in transient signals.
Approach:	They propose a framework that redefines interest fusion as a hierarchical denoising process through diffusion models.
Outcome:	The proposed framework redefines interest fusion as a hierarchical denoising process through diffusion models.

Copied to clipboard

Haoxin Wang, Xianhan Peng, Huang Cheng, Yizhe Huang, Ming Gong, Chenghan Yang, Yang Liu, Jiang Lin

Challenge:	ECom-Bench is a benchmark framework for evaluating LLM agent with multimodal capabilities in e-commerce customer support domain.
Approach:	They introduce a benchmark framework for evaluating LLM agent with multimodal capabilities in the e-commerce customer support domain.
Outcome:	The proposed benchmark features dynamic user simulation based on persona information from real e-commerce customer interactions and a realistic task dataset derived from authentic ecommerce dialogues.

MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction (2022.emnlp-main)

Copied to clipboard

Xiaozhi Wang, Yulin Chen, Ning Ding, Hao Peng, Zimu Wang, Yankai Lin, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, Jie Zhou

MoEfication: Transformer Feed-forward Layers are Mixtures of Experts (2022.findings-acl)

Challenge:	Existing datasets only cover limited relation types at once, which prevents models from taking full advantage of relation interactions.
Approach:	They construct a large-scale human-annotated ERE dataset with improved annotation schemes to address these drawbacks.
Outcome:	The proposed dataset is larger than existing datasets of all the ERE tasks by at least an order of magnitude.

Copied to clipboard

Zhengyan Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

MARCH: Multi-Agent Radiology Clinical Hierarchy for CT Report Generation (2026.acl-short)

Challenge:	Recent work has shown that feed-forward networks (FFNs) in pre-trained Transformers are a key component, storing various linguistic and factual knowledge.
Approach:	They propose to convert a model into its MoE version with the same parameters and build expert routers to decide which experts will be used for each input.
Outcome:	The proposed model can use 10% to 30% of FFN parameters while maintaining over 95% original performance.

Copied to clipboard

Yi Lin, Yihao Ding, Yonghui Wu, Yifan Peng

Text Generation

Challenge:	Automated 3D radiology report generation suffers from clinical hallucinations and lacks the iterative verification characteristic of clinical workflows.
Approach:	They propose a multi-agent framework that emulates the professional hierarchy of radiology departments and assigns specialized roles to distinct agents.
Outcome:	The proposed framework outperforms state-of-the-art models in clinical fidelity and linguistic accuracy on the RadGenome-ChestCT dataset.

Automatic Scene-based Topic Channel Construction System for E-Commerce (2022.emnlp-industry)

Copied to clipboard

Peng Lin, Yanyan Zou, Lingfei Wu, Mian Ma, Zhuoye Ding, Bo Long

Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation (2021.findings-acl)

Challenge:	Recent scene marketing has proved effective for offline shopping.
Approach:	They propose a novel product form, scene-based topic channel, which consists of a list of diverse products belonging to the same usage scenario and a topic title that describes the scenario with marketing words.
Outcome:	The proposed system can be automated and tested on a real-world e-commerce recommendation platform.

Copied to clipboard

Peng Wang, Junyang Lin, An Yang, Chang Zhou, Yichang Zhang, Jingren Zhou, Hongxia Yang

Structured Data in NLP

Challenge:	Existing methods for table-to-text generation suffer from poor faithfulness and low coverage.
Approach:	They propose a method that combines Autoregressive and Non-Autoregressive generation to generate a table-to-text from a key-value table using a skeleton and an edit-based non-autoregressively generation model.
Outcome:	The proposed method outperforms the existing methods on WikiPerson and WikiBio datasets on coverage and faithfulness.

ConvLab-3: A Flexible Dialogue System Toolkit Based on a Unified Data Format (2023.emnlp-demo)

Copied to clipboard

Qi Zhu, Christian Geishauser, Hsien-chin Lin, Carel van Niekerk, Baolin Peng, Zheng Zhang, Shutong Feng, Michael Heck, Nurul Lubis, Dazhen Wan, Xiaochen Zhu, Jianfeng Gao, Milica Gasic, Minlie Huang

Negative-Aware Diffusion Process for Temporal Knowledge Graph Extrapolation (2026.findings-eacl)

Challenge:	Existing tools for building TOD systems often lack a user-friendly interface . a toolkit with advanced, easily integrable modules is needed to bridge this gap .
Approach:	They propose a multifaceted dialogue system toolkit that integrates diverse datasets and models with a streamlined training process and in-depth evaluation tools.
Outcome:	The proposed toolkit combines RL and transfer learning to support the rapid development and evaluation of robust dialogue policies.

Copied to clipboard

Yanglei Gan, Peng He, Yuxiang Cai, Run Lin, Guanyu Zhou, Qiao Liu

On Transferability of Prompt Tuning for Natural Language Processing (2022.naacl-main)

Challenge:	Temporal Knowledge Graphs (TKGs) are dynamic structures representing entities and their evolving relationships through time.
Approach:	They propose a non-parametric model that encodes subject-centric histories into sequential embeddings.
Outcome:	The proposed model encodes subject-centric histories of entities, relations and temporal intervals into sequential embeddings.

Copied to clipboard

Yusheng Su, Xiaozhi Wang, Yujia Qin, Chi-Min Chan, Yankai Lin, Huadong Wang, Kaiyue Wen, Zhiyuan Liu, Peng Li, Juanzi Li, Lei Hou, Maosong Sun, Jie Zhou

Plug-and-Play Knowledge Injection for Pre-trained Language Models (2023.acl-long)

Challenge:	Pre-trained language models (PLMs) can achieve comparable performance to full-parameter fine-tuning by tuning a few soft prompts, but require much more training time than fine-timing.
Approach:	They empirically investigate the transferability of soft prompts across different downstream tasks and PLMs to determine what decides prompt transferability.
Outcome:	The proposed method can achieve comparable performance to full-parameter fine-tuning by tuning a few soft prompts, but requires much more training time than fine-timing.

Copied to clipboard

Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Huadong Wang, Deming Ye, Chaojun Xiao, Xu Han, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learning (2022.findings-emnlp)

Challenge:	Existing knowledge injection methods are not suitable for enhancing pre-trained language models with external knowledge bases.
Approach:	They propose a plug-and-play knowledge injection method where knowledge bases are injected into frozen existing downstream models by a knowledge plugin.
Outcome:	The proposed method improves the performance of knowledge injection on knowledge-driven tasks while keeping model parameters frozen.

Copied to clipboard

Qingyi Si, Yuanxin Liu, Fandong Meng, Zheng Lin, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

PACE: Predictive Adaptive Context Extraction for Long-Horizon LLM Agents (2026.acl-long)

Challenge:	Recent studies have shown that biased samples can be brittle for VQA models . however, the improvements on OOD data severely sacrifice the performance on the in-distribution (ID) data.
Approach:	They propose a contrastive learning approach that exploits biased samples for unbiased information that contributes to reasoning.
Outcome:	The proposed method achieves competitive performance on the OOD dataset while maintaining robustness on the ID dataset.

Copied to clipboard

Lei Wei, Xiao Peng, null Tt, Guannan Zhang, Chenhao Jiang, Hongyu Li, Lanbo Lin, Yuanwu Xu, Jiayao Liu, Kesu Wang, Bin Wang

MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application (2026.acl-long)

Challenge:	Large Language Model (LLM) agents struggle with ultra-long-horizon tasks requiring hundreds or thousands of interaction steps.
Approach:	They propose a framework that reconceptualizes context management as a Next Step Prediction problem.
Outcome:	The proposed framework improves task success rates and robust cross-lingual performance.

Copied to clipboard

Xueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang, Yueru He, Yang Ren, Mingyang Jiang, Vincent Jim Zhang, Yuqing Guo, Jeff Zhao, Huan He, Yi Han, Yun Feng, Yuechen Jiang, Yupeng Cao, Haohang Li, Yangyang Yu, Xiaoyu Wang, Penglei Gao, Shengyuan Lin, Keyi Wang, Shanshan Yang, Yilun Zhao, Zhiwei Liu, Peng Lu, Jerry Huang, Suyuchen Wang, Triantafillos Papadopoulos, Polydoros Giannouris, Efstathia Soufleri, Nuo Chen, Zhiyang Deng, Heming Fu, Yijia Zhao, Mingquan Lin, Meikang Qiu, Kaleb E Smith, Arman Cohan, Xiao-Yang Liu, Jimin Huang, Guojun Xiong, Alejandro Lopez-Lira, Xi Chen, Junichi Tsujii, Jian-Yun Nie, Sophia Ananiadou, Qianqian Xie

RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models (2021.emnlp-main)

Challenge:	Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models.
Approach:	They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts.
Outcome:	The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings .

Copied to clipboard

Wenkai Yang, Yankai Lin, Peng Li, Jie Zhou, Xu Sun

A Simple but Effective Pluggable Entity Lookup Table for Pre-trained Language Models (2022.acl-short)

Challenge:	Backdoor attacks are a serious threat to the safety of reusing deep neural networks (DNNs).
Approach:	They propose an efficient online defense mechanism based on robustness-aware perturbations to distinguish poisoned and clean samples to defend against backdoor attacks on natural language processing models.
Outcome:	The proposed method achieves better defending performance and lower computational costs than existing defense methods.

Copied to clipboard

Deming Ye, Yankai Lin, Peng Li, Maosong Sun, Zhiyuan Liu

Strong Baselines for Simple Question Answering over Knowledge Graphs with and without Neural Networks (N18-2)

Challenge:	Existing pre-trained language models cannot recall factual knowledge of entities exhibited in large-scale corpora, especially those rare entities.
Approach:	They propose to build a pluggable Entity Lookup Table (PELT) on demand by aggregating the entity’s output representations of multiple occurrences in the corpora.
Outcome:	The proposed model can transfer entity knowledge from out-of-domain corpora into PLMs with different architectures.

Copied to clipboard

Salman Mohammed, Peng Shi, Jimmy Lin

NumNet: Machine Reading Comprehension with Numerical Reasoning (D19-1)

Challenge:	Existing work on simple question answering over knowledge graphs involves increasingly complex NN architectures.
Approach:	They propose to decompose the problem into entity detection, entity linking, relation prediction, evidence combination and heuristics.
Outcome:	The proposed approach outperforms existing models and benchmarks on a simple QA task.

Copied to clipboard

Qiu Ran, Yankai Lin, Peng Li, Jie Zhou, Zhiyuan Liu

Reasoning

Challenge:	Existing numerical MRC models are weak in numerical reasoning, such as addition, subtraction, sorting and counting.
Approach:	They propose a numerical MRC model that integrates numerical reasoning into existing MRC models and achieves an EM-score of 64.56% on the DROP dataset.
Outcome:	The proposed model outperforms all existing machine reading comprehension models by considering the numerical relations among numbers on the DROP dataset.

ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models (2024.findings-acl)

Copied to clipboard

Haoran Luo, Haihong E, Zichen Tang, Shiyao Peng, Yikai Guo, Wentai Zhang, Chenghao Ma, Guanting Dong, Meina Song, Wei Lin, Yifan Zhu, Anh Tuan Luu

Meta-Transfer Learning for Code-Switched Speech Recognition (2020.acl-main)

Challenge:	Existing KBQA methods address inefficient knowledge retrieval and semantic parsing errors.
Approach:	They propose a generatethen-retrieve KBQA framework that generates logical form and replaces entities and relations with an unsupervised retrieval method to improve both generation and retrieval more directly.
Outcome:	Experimental results show that ChatKBQA achieves new state-of-the-art performance on standard KBQA datasets, WebQSP, and CWQ.

Copied to clipboard

Genta Indra Winata, Samuel Cahyawijaya, Zhaojiang Lin, Zihan Liu, Peng Xu, Pascale Fung

Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Study (2025.coling-main)

Challenge:	Increasing number of people in the world today speak a mixed-language as a result of being multilingual.
Approach:	They propose a method to transfer learn on a code-switched speech recognition system by extracting information from high-resource monolingual datasets.
Outcome:	The proposed model outperforms baselines on speech recognition and language modeling tasks and is faster to converge.

Copied to clipboard

Bowen Li, Wenhan Wu, Ziwei Tang, Lin Shi, John Yang, Jinyang Li, Shunyu Yao, Chen Qian, Binyuan Hui, Qicheng Zhang, Zhiyin Yu, He Du, Ping Yang, Dahua Lin, Chao Peng, Kai Chen

LaMPE: Length-aware Multi-grained Positional Encoding for Adaptive Long-context Scaling Without Training (2026.findings-acl)

Challenge:	Existing benchmarks focused on simplified or isolated aspects of coding, ignoring the full spectrum of programming challenges.
Approach:	They propose a case study that examines the performance of large language models across the entire software development lifecycle with four programming languages, multiple domains, and carefully designed and verified metrics for each task.
Outcome:	The proposed model performs across the entire software development lifecycle, including design, environment setup, implementation, acceptance testing, and unit testing.

Copied to clipboard

Sikui Zhang, Guangze Gao, Ziyun Gan, Chunfeng Yuan, Zefeng Lin, Houwen Peng, Bing Li, Weiming Hu

Challenge:	Large language models (LLMs) experience significant performance degradation when the input exceeds the pretraining context window due to the out-of-distribution (OOD) behavior of Rotary Position Embedding (RoPE).
Approach:	They propose a training-free method that remaps out-of-distribution (OOD) positions into the in-distance range with fixed mapping strategies, ignoring the dynamic relationship between input length and effective context window.
Outcome:	Experiments on three representative LLMs across five mainstream long-context benchmarks show that the proposed method achieves significant performance improvements compared to existing methods.

Simple Attention-Based Representation Learning for Ranking Short Social Media Posts (N19-1)

Copied to clipboard

Peng Shi, Jinfeng Rao, Jimmy Lin

Representation Learning

Challenge:	Existing approaches to ranking short social media posts are complex and require different components to capture a multitude of relevance signals.
Approach:	They propose a word-level Siamese architecture with attention-based mechanisms for capturing semantic "soft" matches between query and post tokens.
Outcome:	The proposed model is faster and simpler than existing models and more efficient than existing approaches.

Do Pre-trained Models Benefit Knowledge Graph Completion? A Reliable Evaluation and a Reasonable Approach (2022.findings-acl)

Copied to clipboard

Xin Lv, Yankai Lin, Yixin Cao, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, Jie Zhou

Manual Evaluation Matters: Reviewing Test Protocols of Distantly Supervised Relation Extraction (2021.findings-acl)

Challenge:	Pre-trained language models capture factual knowledge from massive texts . but they are still quite behind the SOTA KGC models in terms of performance .
Approach:	They propose to use open-world assumption to evaluate PLM-based knowledge graph completion models . they propose to convert each triple and its support information into natural prompt sentences .
Outcome:	The proposed model is more accurate under the open-world assumption (OWA) this setting manual checks the correctness of knowledge that is not in KGs.

Copied to clipboard

Tianyu Gao, Xu Han, Yuzhuo Bai, Keyue Qiu, Zhiyu Xie, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Learning to Recover from Multi-Modality Errors for Non-Autoregressive Neural Machine Translation (2020.acl-main)

Challenge:	Distantly supervised relation extraction (RE) has attracted much attention in the past few years . previous methods to evaluate models manually or directly on autolabeled data have produced inaccurate evaluations .
Approach:	They propose to use distant supervision to generate large-scale autolabeled data . they build manually-annotated test sets for two DS-RE datasets and evaluate models .
Outcome:	The proposed method produces 53% wrong labels at the entity pair level in the popular NYT10 dataset.

Copied to clipboard

Qiu Ran, Yankai Lin, Peng Li, Jie Zhou

Web Fraud Attacks Against LLM-Driven Multi-Agent Systems (2026.findings-acl)

Challenge:	Existing non-autoregressive neural machine translation models suffer from multi-modality problem . despite their autoregressivity, most NMT models suffer with slow decoding speed .
Approach:	They propose a semi-autoregressive model which generates a translation as a sequence of segments while each segment is predicted token-by-token.
Outcome:	The proposed model can achieve 4 times speedup while maintaining comparable performance.

Copied to clipboard

Dezhang Kong, Hujin Peng, Yilun Zhang, Lele Zhao, Zhenhua Xu, Shi Lin, Changting Lin, Meng Han

XRICL: Cross-lingual Retrieval-Augmented In-Context Learning for Cross-lingual Text-to-SQL Semantic Parsing (2022.findings-emnlp)

Challenge:	Large Language Model (LLM)-driven multi-agent systems (MAS) are rapidly gaining popularity, and its inherent security risks are rapidly becoming a concern.
Approach:	They propose a novel attack manipulating unique structures of web links to deceive MAS by using homoglyph deception, sub-directory nesting, and parameter obfuscation.
Outcome:	The proposed attacks exploit unique structures of web links to deceive MAS . they exhibit significant destructive potential across different MAS architectures .

Copied to clipboard

Peng Shi, Rui Zhang, He Bai, Jimmy Lin

Learning to Win Lottery Tickets in BERT Transfer via Task-agnostic Mask Training (2022.naacl-main)

Challenge:	Existing work focuses on English datasets, and it is unclear whether large language models can serve as competitive semantic parsers for other languages.
Approach:	They propose a framework that learns to retrieve relevant English exemplars for a given query to construct prompts.
Outcome:	The proposed framework learns to retrieve relevant English exemplars for a given query to construct prompts.

Copied to clipboard

Yuanxin Liu, Fandong Meng, Zheng Lin, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts (2025.acl-long)

Challenge:	Recent studies show pre-trained language models contain matching subnetworks that have similar transfer learning performance as the original PLM.
Approach:	They propose to prune matching subnetworks using magnitude-based pruning . they propose to optimize the subnetwork structure towards the pre-training objectives .
Outcome:	The proposed method is more efficient in searching subnetworks and advantageous when fine-tuning within a range of data scarcity.

Copied to clipboard

Yuchen Feng, Bowen Shen, Naibin Gu, Jiaxuan Zhao, Peng Fu, Zheng Lin, Weiping Wang

Rethinking Stealthiness of Backdoor Attack against NLP Models (2021.acl-long)

Challenge:	Existing methods for reconstruction of large language models overlook diversity among experts, leading to potential redundancy.
Approach:	They propose a pruning-based expert reconstruction method that prunes a specific LLM and retrains it on routers, experts and normalization modules.
Outcome:	The proposed method outperforms pruning and MoE reconstruction methods on Llama-style models with open-source training corpora.

Copied to clipboard

Wenkai Yang, Yankai Lin, Peng Li, Jie Zhou, Xu Sun

PwnGPT: Automatic Exploit Generation Based on Large Language Models (2025.acl-long)

Challenge:	Existing backdoor attacks are not stealthy to system deployers or users.
Approach:	They propose a novel backdoor attack method based on negative data augmentation and modifying word embeddings that is much stealthier while maintaining pretty good attacking performance.
Outcome:	The proposed method is much stealthier while maintaining pretty good attacking performance.

Copied to clipboard

Wanzong Peng, Lin Ye, Xuetao Du, Hongli Zhang, Dongyang Zhan, Yunting Zhang, Yicheng Guo, Chen Zhang

Prompt Tuning for Unified Multimodal Pretrained Models (2023.findings-acl)

Challenge:	Automated exploit generation (AEG) is the automatic discovery and exploitation of vulnerabilities against unknown targets.
Approach:	They propose an automatic exploit generation framework that automatically solves pwn challenges by using large language models.
Outcome:	The proposed framework improves the completion rate of exploits on the openAI o1-preview model and the GPT-4o model.

Copied to clipboard

Hao Yang, Junyang Lin, An Yang, Peng Wang, Chang Zhou

Fully Hyperbolic Neural Networks (2022.acl-long)

Challenge:	Prompt tuning has demonstrated success in natural language pretraining and even vision pretraining.
Approach:	They propose to apply prompt tuning to a unified sequence-to-sequence pretrained model by adding a sequence of learnable embeddings to each layer and finetuning the pretrained models on downstream tasks.
Outcome:	The proposed method outperforms other parameter-efficient tuning methods on multimodal models and is robust against adversarial attacks.

Copied to clipboard

Weize Chen, Xu Han, Yankai Lin, Hexu Zhao, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Challenge:	Existing hyperbolic neural networks encode features in the hyperbolical space yet formalize most of their operations in the tangent space.
Approach:	They propose a fully hyperbolic framework to build hyperbolical networks based on the Lorentz model by adapting Lorentzer transformations to formalize essential operations of neural networks.
Outcome:	The proposed framework has better performance on four NLP tasks compared with existing hyperbolic models .

Question-Interlocutor Scope Realized Graph Modeling over Key Utterances for Dialogue Reading Comprehension (2023.findings-acl)

Copied to clipboard

Jiangnan Li, Mo Yu, Fandong Meng, Zheng Lin, Peng Fu, Weiping Wang, Jie Zhou

Qwen2.5-xCoder: Multi-Agent Collaboration for Multilingual Code Instruction Tuning (2025.acl-long)

Challenge:	Compared to standard RC tasks, dialogue reading comprehension (DRC) has raised challenges because of the complex speaker information and noisy dialogue context.
Approach:	They propose a new method for dialogue reading comprehension that extracts answers from dialogues by using key-utterances-extracting methods and a Question-Interlocutor Scope Realized Graph.
Outcome:	The proposed method achieves state-of-the-art performance against previous works.

Copied to clipboard

Jian Yang, Wei Zhang, Yibo Miao, Shanghaoran Quan, Zhenhe Wu, Qiyao Peng, Liqun Yang, Tianyu Liu, Zeyu Cui, Binyuan Hui, Junyang Lin

Half-S: Halving the Scale for Near-Lossless 4-Bit LLM Training (2026.findings-acl)

Challenge:	Existing methods to train code LLMs view each programming language in isolation . experimental results show that Qwen2.5-xCoder can bridge the gap between different programming languages .
Approach:	They propose a framework that allows agents to collaborate to enhance multilingual instruction tuning for code LLMs.
Outcome:	Experimental results show that Qwen2.5-xCoder can transfer knowledge efficiently and effectively between languages.

Copied to clipboard

Jinyang Du, Ruihao Gong, Linghan Ai, Zining Wang, Yunke Peng, Yao Wang, Lei Yan, null Wxuefei, Yaoyuan Wang, Jinyang Guo, Dahua Lin, Xianglong Liu

Green & Sustainable NLP

Challenge:	Existing 4-bit training pipelines rely on max-scaling, which causes representation collapse . despite this, there are limitations in the accuracy of 4-bit LLM training .
Approach:	They propose a scaling strategy that uses half-scaling as a hardware-friendly default . they propose fp4 support that allows for a faster scaling of large language models .
Outcome:	The proposed scaling strategy narrows the gap between theoretical optimum and BF16 while maintaining the efficiency benefits of 4-bit training.

Cross-Lingual Word Embedding Refinement by ℓ1 Norm Optimisation (2021.naacl-main)

Copied to clipboard

Xutan Peng, Chenghua Lin, Mark Stevenson

Representation Learning

Challenge:	Existing methods for building high-quality CLWEs learn mappings that minimise the l2 norm loss function but this optimisation objective has been shown to be sensitive to outliers.
Approach:	They propose a simple post-processing step to improve cross-lingual word embeddings using the Manhattan norm goodness-of-fit criterion.
Outcome:	The proposed approach outperforms four state-of-the-art baselines in bilingual lexicon induction and cross-lingual transfer tasks.

ROSE: Robust Selective Fine-tuning for Pre-trained Language Models (2022.emnlp-main)

Copied to clipboard

Lan Jiang, Hao Zhou, Yankai Lin, Peng Li, Jie Zhou, Rui Jiang

Check It Again:Progressive Visual Question Answering via Visual Entailment (2021.acl-long)

Challenge:	Recent studies have highlighted the lack of adversarial robustness in pre-trained models.
Approach:	They propose a fine-tuning approach that conducts selective updates when adapting pre-trained models to downstream tasks.
Outcome:	The proposed approach improves adversarial robustness on downstream tasks . it eliminates spurious updates, leading to flatter and wider optima than the conventional method .

Copied to clipboard

Qingyi Si, Zheng Lin, Ming yu Zheng, Peng Fu, Weiping Wang

Visual Data in NLP

Challenge:	Existing approaches to Visual Question Answering (VQA) only address superficial correlations between image and answer.
Approach:	They propose a select-and-rerank progressive framework based on Visual Entailment to address this problem.
Outcome:	The proposed framework improves on the Visual Question Answering (VQA) task with 7.55% accuracy.

SPEAK: Spiking Neurons as an Entropy-Aware Tokenizer for Large Language Models (2026.acl-long)

Copied to clipboard

Ming Chen, Wenyao Li, Chao Liang, Shi Gu, Peng Lin, De Ma, Huajin Tang, Qian Zheng, Gang Pan

From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models (2022.findings-emnlp)

Challenge:	Existing tokenizers fail to explicitly leverage historical tokenization results . large language models (LLMs) have demonstrated remarkable effectiveness across NLP tasks .
Approach:	They propose a tokenizer that integrates spiking neurons to explicitly leverage historical tokenization results.
Outcome:	The proposed tokenizer leverages historical tokenization results, but does not selectively leverage history based on contextual relevance.

Copied to clipboard

Lei Li, Yankai Lin, Xuancheng Ren, Guangxiang Zhao, Peng Li, Jie Zhou, Xu Sun

Multimodal Transformers are Hierarchical Modal-wise Heterogeneous Graphs (2025.acl-long)

Challenge:	Existing models for natural language processing (NLP) are fine-tuned and released for research and deployments.
Approach:	They propose a PLM reuse paradigm that merges teacher-PLM knowledge into a student model.
Outcome:	The proposed paradigm can reduce the computational cost and environmental side-effects of retraining the PLM from scratch.

Copied to clipboard

Yijie Jin, Junjie Peng, Xuanchao Lin, Haochen Yuan, Lan Wang, Cangzhi Zheng

ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning (2021.acl-long)

Challenge:	Multimodal Sentiment Analysis (MSA) is a rapidly developing field that integrates multimodal information to recognize sentiments.
Approach:	They propose a multimodal fusion model that integrates multimodal information to recognize sentiments using multimodal transformers.
Outcome:	The proposed model achieves significantly higher performance than MulTs and the existing model is robust.

Copied to clipboard

Yujia Qin, Yankai Lin, Ryuichi Takanobu, Zhiyuan Liu, Peng Li, Heng Ji, Minlie Huang, Maosong Sun, Jie Zhou

CodRED: A Cross-Document Relation Extraction Dataset for Acquiring Knowledge in the Wild (2021.emnlp-main)

Challenge:	Existing pre-training objectives do not explicitly model relational facts in text . Experimental results show that ERICA can improve typical PLMs on several language understanding tasks, including relation extraction, entity typing and question answering.
Approach:	They propose a contrastive learning framework ERICA to obtain a deep understanding of entities and relations in text.
Outcome:	The proposed framework can improve PLMs on several language understanding tasks, especially under low-resource settings.

Copied to clipboard

Yuan Yao, Jiaju Du, Yankai Lin, Peng Li, Zhiyuan Liu, Jie Zhou, Maosong Sun

Target Really Matters: Target-aware Contrastive Learning and Consistency Regularization for Few-shot Stance Detection (2022.coling-1)

Challenge:	Existing relation extraction methods focus on extracting relational facts between entity pairs within single sentences or documents.
Approach:	They present a problem of cross-document relation extraction (CRE) using human annotations.
Outcome:	The proposed dataset is the first human-annotated cross-document RE dataset . it shows that it is challenging to existing RE methods including strong BERT-based models.

Copied to clipboard

Rui Liu, Zheng Lin, Huishan Ji, Jiangnan Li, Peng Fu, Weiping Wang

Modeling Intra and Inter-modality Incongruity for Multi-Modal Sarcasm Detection (2020.findings-emnlp)

Challenge:	stance detection is a task to identify attitudes from opinions towards certain targets, but it is expensive and time-consuming . stance detector is based on labeled data, but unlabeled data can be collected easier .
Approach:	They propose a semi-supervised framework for few-shot stance detection that uses unlabeled data to learn more distinguishable representations for different targets.
Outcome:	The proposed framework achieves state-of-the-art performance on multiple benchmark datasets.

Copied to clipboard

Hongliang Pan, Zheng Lin, Peng Fu, Yatao Qi, Weiping Wang

Sentiment Analysis

Challenge:	Existing methods for sarcasm detection ignore the incongruity character in sarcasm, which is often manifested between modalities or within modalités.
Approach:	They propose to capture inter-modality incongruity in a text-based model by using a self-attention mechanism and a co-attention model to model the contradiction within the text.
Outcome:	The proposed model achieves state-of-the-art on a public multi-modal sarcasm detection dataset.

A Scalable Multi-LLM Collaboration System with Retrieval-based Selection and Exploration-Exploitation-Driven Enhancement (2026.acl-long)

Copied to clipboard

Shengji Tang, Jianjian Cao, Weihao Lin, Jiale Hong, Bo Zhang, Shuyue Hu, Lei Bai, Tao Chen, Wanli Ouyang, Peng Ye

A Gradient Control Method for Backdoor Attacks on Parameter-Efficient Tuning (2023.acl-long)

Challenge:	Existing multi-LLM collaboration systems often encounter scalability challenges when integrating new LLMs and tasks.
Approach:	They propose a Scalable Multi-LLM Collaboration System to coordinate multiple open-source LLMs.
Outcome:	The proposed system outperforms prevailing closed-source LLMs on eight mainstream benchmarks on multiple tasks.

Copied to clipboard

Naibin Gu, Peng Fu, Xiyu Liu, Zhengxiao Liu, Zheng Lin, Weiping Wang

Aligning Cross-Lingual Entities with Multi-Aspect Information (D19-1)

Challenge:	Parameter-Efficient Tuning (PET) fine-tunes pre-trained language models for downstream tasks, but a large reduction in the number of attackable parameters will greatly affect the effectiveness of backdoor attacks, resulting in backdoor forgetting.
Approach:	They propose a gradient control method to consolidate the attack effect by freezing most parameters of the pre-trained model and fine-tuning only a small number of parameters.
Outcome:	The proposed method improves sentiment classification and spam detection, and can be applied to different tasks.

Copied to clipboard

Hsiu-Wei Yang, Yanyan Zou, Peng Shi, Wei Lu, Jimmy Lin, Xu Sun

Adapt Once, Thrive with Updates: Transferable Parameter-Efficient Fine-Tuning on Evolving Base Models (2025.acl-long)

Challenge:	Existing knowledge graphs that represent entities in different languages are not covered by existing systems.
Approach:	They propose two ways to embed entities from multilingual knowledge graphs into the same vector space, where equivalent entities are close to each other.
Outcome:	The proposed method significantly outperforms existing systems on two benchmark datasets.

Copied to clipboard

Naibin Gu, Peng Fu, Xiyu Liu, Ke Ma, Zheng Lin, Weiping Wang

SolEval: Benchmarking Large Language Models for Repository-level Solidity Smart Contract Generation (2025.emnlp-main)

Challenge:	Parameter-efficient fine-tuning (PEFT) is a common method for fine- tuning large language models . however, once updated, PEFT modules suffer performance degradation on newer versions .
Approach:	They propose a method that enhances the PEFT module by focusing on the task-specific pattern while reducing its dependence on certain knowledge in the base model.
Outcome:	Experiments show that PEFT modules can maintain performance on updated models without re-tuning . the proposed approach can be used in real-world applications with large model sizes .

Copied to clipboard

Zhiyuan Peng, Xin Yin, Rui Qian, Peiqin Lin, YongKang Liu, Hao Zhang, Chenhao Ying, Yuan Luo

Continual Relation Learning via Episodic Memory Activation and Reconsolidation (2020.acl-main)

Challenge:	Existing methods focus on Python and Java, neglecting Solidity, the programming language for Ethereum smart contracts.
Approach:	They construct a repository-level benchmark for Solidity to evaluate the performance of LLMs on Ethereum.
Outcome:	The proposed benchmarks show that the best performing LLM achieves only 26.29% Pass@10, highlighting room for improvement in Solidity code generation.

Copied to clipboard

Xu Han, Yi Dai, Tianyu Gao, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Challenge:	Existing methods to learn incessantly emerging novel relations are overfitting the few memorized examples of old relations, causing confusion among existing relations.
Approach:	They introduce episodic memory activation and reconsolidation (EMAR) to continual relation learning.
Outcome:	The proposed method outperforms state-of-the-art models in catastrophic forgetting old relations.

Automatic Marketing Theme and Commodity Construction System for E-commerce (2023.emnlp-industry)

Copied to clipboard

Zhiping Wang, Peng Lin, Hainan Zhang, Hongshen Chen, Tianhao Li, Zhuoye Ding, Sulong Xu, Jinghe Hu

HSS-Synth: Humanities and Social Sciences Data Synthesis for LLMs (2026.findings-acl)

Challenge:	Existing recommendation system invites experts to write marketing themes and select relevant commodities, which suffer from difficulty in mass production, poor timeliness and low online indicators.
Approach:	They propose to use pretrained language model to generate marketing themes and commodity consistency module to select relevant commodities for the generative theme.
Outcome:	The proposed system can generate popular marketing themes and select relevant commodities automatically and improve theme online effectiveness compared with state-of-the-art methods.

Copied to clipboard

Ru Peng, Tianyu Zhao, Xijun Gu, Zhiting Fan, Haokai Xu, Jinyang Zhang, Yawen Zeng, Yihong Zhuang, Kexin Yang, Junyang Lin, Dayiheng Liu, Junbo Zhao

Challenge:	High-quality, diverse data are vital for large language models (LLMs) but remain scarce and costly.
Approach:	They define the first HSS domain system covering 14 mainstream fields and introduce HSS-Synth.
Outcome:	the proposed pipeline outperforms 14 leading baselines on 16 benchmarks.

Filter-then-Generate: Large Language Models with Structure-Text Adapter for Knowledge Graph Completion (2025.coling-main)

Copied to clipboard

Ben Liu, Jihai Zhang, Fangquan Lin, Cheng Yang, Min Peng

Cross-lingual Spoken Language Understanding with Regularized Representation Alignment (2020.emnlp-main)

Challenge:	Empirical evidence suggests that LLMs perform worse than conventional KGC approaches.
Approach:	They propose a filter-then-generate paradigm and a multiple-choice question format to harness the capability of LLMs while mitigating the issue casused by hallucinations.
Outcome:	The proposed method achieves substantial performance gain compared to existing state-of-the-art methods.

Copied to clipboard

Zihan Liu, Genta Indra Winata, Peng Xu, Zhaojiang Lin, Pascale Fung

BeamLoRA: Beam-Constraint Low-Rank Adaptation (2025.acl-long)

Challenge:	despite promising results, current cross-lingual models suffer from imperfect cross-linguistic representation alignments between the source and target languages, which makes the performance sub-optimal.
Approach:	They propose a regularization approach to align word-level and sentence-level representations across languages without external resources.
Outcome:	The proposed model outperforms state-of-the-art models in few-shot and zero-shot scenarios and achieves comparable performance to supervised training with all training data.

Copied to clipboard

Naibin Gu, Zhenyu Zhang, Xiyu Liu, Peng Fu, Zheng Lin, Shuohuan Wang, Yu Sun, Hua Wu, Weiping Wang, Haifeng Wang

Reasoning

Challenge:	Low-Rank Adaptation (LoRA) is one of the most efficient parameter-efficient fine-tuning methods.
Approach:	They propose to conceptualize each LoRA module as a beam where each rank corresponds to a potential sub-solution.
Outcome:	The proposed method improves performance on three base models and 12 datasets.

ELLE: Efficient Lifelong Pre-training for Emerging Data (2022.findings-acl)

Copied to clipboard

Yujia Qin, Jiajie Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Learning from Context or Names? An Empirical Study on Neural Relation Extraction (2020.emnlp-main)

Challenge:	Existing pre-trained language models are typically trained with static data, ignoring that streaming data of various sources may continuously grow.
Approach:	They propose to use function preserved model expansion to expand existing PLM's width and depth to improve efficiency of knowledge acquisition.
Outcome:	The proposed model improves pre-training efficiency and performance over existing models on BERT and GPT.

Copied to clipboard

Hao Peng, Tianyu Gao, Xu Han, Yankai Lin, Peng Li, Zhiyuan Liu, Maosong Sun, Jie Zhou

Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQA (2022.findings-emnlp)

Challenge:	Existing datasets may leak shallow heuristics via entity mentions, thus contributing to the high performance on RE benchmarks.
Approach:	They propose an entity-masked contrastive framework for relation extraction to gain a deeper understanding on textual context and type information while avoiding rote memorization of entities.
Outcome:	The proposed framework improves the effectiveness and robustness of neural models in different RE scenarios.

Copied to clipboard

Qingyi Si, Fandong Meng, Mingyu Zheng, Zheng Lin, Yuanxin Liu, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

Question Answering

Challenge:	Visual Question Answering (VQA) models are prone to learn the shortcut solution formed by dataset biases rather than the intended solution.
Approach:	They propose a dataset that considers varying types of shortcuts by constructing different distribution shifts in multiple OOD test sets.
Outcome:	The proposed dataset considers varying types of shortcuts by constructing different distribution shifts in multiple OOD test sets.

Transferring General Multimodal Pretrained Models to Text Recognition (2023.findings-acl)

Copied to clipboard

Junyang Lin, Xuancheng Ren, Yichang Zhang, Gao Liu, Peng Wang, An Yang, Chang Zhou

Compressing and Debiasing Vision-Language Pre-Trained Models for Visual Question Answering (2023.emnlp-main)

Challenge:	Existing methods for text recognition rely on large-scale pretraining on human-annotated or synthetic data.
Approach:	They propose a method to transfer multimodal pretrained models to text recognition using image captioning.
Outcome:	The proposed method outperforms the baselines and achieves state-of-the-art performance in the Chinese text recognition benchmark.

Copied to clipboard

Qingyi Si, Yuanxin Liu, Zheng Lin, Peng Fu, Yanan Cao, Weiping Wang

Dynamic Knowledge Distillation for Pre-trained Language Models (2021.emnlp-main)

Challenge:	Existing studies on VQA models have found that they suffer from dataset biases and inefficient memory footprints.
Approach:	They investigate whether a VLP can be compressed and debiased simultaneously by searching sparse and robust subnetworks.
Outcome:	The proposed compression and debiasing pipelines outperform the debiased full VLPs on VQA tasks.

Copied to clipboard

Lei Li, Yankai Lin, Shuhuai Ren, Peng Li, Jie Zhou, Xu Sun

CBP-Tuning: Efficient Local Customization for Black-box Large Language Models (2025.emnlp-main)

Challenge:	Existing methods conduct knowledge distillation statically, e.g., student model aligns output distribution to teacher model on pre-defined training dataset.
Approach:	They propose a dynamic knowledge distillation that empowers the student to adjust the learning procedure according to its competency . they find it is promising and provide discussions on potential future directions towards more efficient methods .
Outcome:	The proposed method can boost student model performance while accelerating training . the proposed method reduces memory usage and accelerates model inference .

Copied to clipboard

Jiaxuan Zhao, Naibin Gu, Yuchen Feng, Xiyu Liu, Peng Fu, Zheng Lin, Weiping Wang

CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade (2021.findings-emnlp)

Challenge:	Customized black-box prompt tuning is a new approach to customize large language models . however, as models grow, the resources required for training and deployment become increasingly expensive .
Approach:	They propose a framework that facilitates efficient local customization while preserving bidirectional privacy.
Outcome:	The proposed framework facilitates efficient local customization while preserving bidirectional privacy.

Copied to clipboard

Lei Li, Yankai Lin, Deli Chen, Shuhuai Ren, Peng Li, Jie Zhou, Xu Sun

Answering Complex Open-domain Questions Through Iterative Query Generation (D19-1)

Challenge:	Experimental results show that CascadeBERT can achieve an overall 15% improvement under 4x speed-up compared with existing dynamic early exiting methods on six classification tasks.
Approach:	They propose a framework which emits predictions in internal layers without passing through the entire model.
Outcome:	The proposed framework can achieve 15% improvement under 4x speed-up compared with existing methods on six classification tasks yielding more calibrated and accurate predictions.

Copied to clipboard

Peng Qi, Xiaowen Lin, Leo Mehr, Zijian Wang, Christopher D. Manning

DocRED: A Large-Scale Document-Level Relation Extraction Dataset (P19-1)

Challenge:	Currently, one-step retrieve-and-read question answering systems cannot answer such questions because they rarely contain retrievable clues about the missing entity.
Approach:	They propose a multi-step approach to retrieve relevant content with the question, then reading the paragraphs returned by the information retrieval component to arrive at the final answer.
Outcome:	The proposed model outperforms the best previously published model despite not using pretrained language models such as BERT.

Copied to clipboard

Yuan Yao, Deming Ye, Peng Li, Xu Han, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Lixin Huang, Jie Zhou, Maosong Sun