Papers by Peng Zhao
Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are trained on vast corpora that contain substantial knowledge but their outputs often contain confidently stated inaccuracies. |
| Approach: | They propose to encode truthfulness as a distinct linear feature, termed the "truth direction", which can classify truthfulness reliably. |
| Outcome: | The proposed model can generalize to logical transformations, question-answering tasks, in-context learning, and external knowledge sources. |
On the Analysis and Distillation of Emergent Outlier Properties in Pre-trained Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies show that a small subset of dimensions within language Transformers’ representation spaces emerge as "outliers" during pretraining. |
| Approach: | They propose a method that prioritizes critical outlier dimensions in distillation using a weighted MSE loss. |
| Outcome: | The proposed method outperforms state-of-the-art distillation methods and generalizes well across Encoder-only BERT, Decoder-only GPT-2, and Encodeer-Decoder T5 architectures. |
M3Seg: A Maximum-Minimum Mutual Information Paradigm for Unsupervised Topic Segmentation in ASR Transcripts (2023.emnlp-main)
Copied to clipboard
| Challenge: | Topic segmentation aims to split automatic speech recognition transcriptions into segments that are bounded by thematic meanings. |
| Approach: | They propose a Maximum-Minimum Mutual information paradigm for linear topic segmentation without using any parallel data. |
| Outcome: | The proposed paradigm outperforms the state-of-the-art methods by a significant margin. |
RE-Matching: A Fine-Grained Semantic Matching Method for Zero-Shot Relation Extraction (2023.acl-long)
Copied to clipboard
Jun Zhao, WenYu Zhan, Xin Zhao, Qi Zhang, Tao Gui, Zhongyu Wei, Junzhe Wang, Minlong Peng, Mingming Sun
| Challenge: | Existing methods for zero-shot relation extraction lack explicit modeling of matching pattern . et al. (2018) show that our method achieves higher matching accuracy and faster inference speed . |
| Approach: | They propose a fine-grained semantic matching method tailored for zero-shot relation extraction . they decompose sentence-level similarity score into entity matching score and context matching score . |
| Outcome: | The proposed method achieves higher matching accuracy and faster inference speed than state-of-the-art methods. |
Lil: Less is Less When Applying Post-Training Sparse-Attention Algorithms in Long-Decode Stage (2026.findings-acl)
Copied to clipboard
Junhao Hu, Fangze Li, Mingtao Xu, Feifan Meng, Shiju Zhao, Tiancheng Hu, Ting Peng, Anmin Liu, Wenrui Huang, Chenxu Liu, Ziyue Hua, Tao Xie
| Challenge: | Prior work typically decomposes inference into prefill and decode stages, with the decode stage dominating total latency. |
| Approach: | They propose an algorithm that detects threshold where information loss exceeds information gain during sparse decoding to reduce token consumption by up to 90% and a marginal accuracy degradation of less than 2%. |
| Outcome: | The proposed algorithm reduces token consumption by 90% with a marginal accuracy degradation of less than 2% across reasoning-intensive benchmarks. |
Self-supervised Quantized Representation for Seamlessly Integrating Knowledge Graphs with Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are gaining popularity due to their lack of knowledge hallucination and lack of a coherent model. |
| Approach: | They propose a self-supervised quantized representation method to compress KG structural and semantic knowledge into discrete codes that align the format of language sentences. |
| Outcome: | The proposed framework outperforms existing unsupervised methods producing more distinguishable codes on KG link prediction and triple classification tasks. |
FOLIO: Natural Language Reasoning with First-Order Logic (2024.emnlp-main)
Copied to clipboard
Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, Lucy Sun, Alexander Wardle-Solano, Hannah Szabó, Ekaterina Zubova, Matthew Burtell, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kasai, Tao Yu, Rui Zhang, Alexander Fabbri, Wojciech Kryscinski, Semih Yavuz, Ye Liu, Xi Lin, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Rex Ying, Arman Cohan, Dragomir Radev
| Challenge: | Existing benchmarks for logical reasoning in large language models lack language naturalness or limited complexity. |
| Approach: | They propose to use first-order logic annotations to evaluate logical reasoning capabilities of large language models. |
| Outcome: | The proposed dataset evaluates the FOL reasoning ability of supervised fine-tuning on medium-sized language models. |
End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Grounding (2022.acl-long)
Copied to clipboard
Mengze Li, Tianbao Wang, Haoyu Zhang, Shengyu Zhang, Zhou Zhao, Jiaxu Miao, Wenqiao Zhang, Wenming Tan, Jin Wang, Peng Wang, Shiliang Pu, Fei Wu
| Challenge: | Existing methods for grounding video frames with dense annotations require enormous amount of human effort. |
| Approach: | They propose to ground natural language in video frames with only one frame labeled . they propose an end-to-end model that eliminates interference of irrelevant frames . |
| Outcome: | The proposed model can ground natural language in all video frames with only one frame labeled . the proposed model eliminates interference of irrelevant frames based on branch search and cropping techniques . |
Mitigating Bias for Question Answering Models by Tracking Bias Influence (2024.naacl-long)
Copied to clipboard
Mingyu Ma, Jiun-Yu Kao, Arpit Gupta, Yu-Hsiang Lin, Wenbo Zhao, Tagyoung Chung, Wei Wang, Kai-Wei Chang, Nanyun Peng
| Challenge: | Existing literature observes bias in question answering (QA) models, but there is no method to mitigate it. |
| Approach: | They propose an approach to mitigate the bias of question answering models by observing the influence of a query instance on another instance. |
| Outcome: | The proposed method reduces bias level in all 9 bias categories while maintaining comparable QA accuracy. |
SocAoG: Incremental Graph Parsing for Social Relation Inference in Dialogues (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on identifying entities' relations from the semantics of dialogues-they utilize either the attention mechanism or a refined token graph to locate informative words. |
| Approach: | They propose a sequential structure prediction task to incrementally parse SocAoG for dynamic inference upon any incoming utterance. |
| Outcome: | Empirical results show that the proposed model infers social relations more accurately than the state-of-the-art methods. |
AgentBank: Towards Generalized LLM Agents via Fine-Tuning on 50000+ Interaction Trajectories (2024.findings-emnlp)
Copied to clipboard
Yifan Song, Weimin Xiong, Xiutian Zhao, Dawei Zhu, Wenhao Wu, Ke Wang, Cheng Li, Wei Peng, Sujian Li
| Challenge: | Existing studies focus on specialized agents designed for particular tasks. |
| Approach: | They propose to scale annotated interaction trajectories and fine-tune LLMs on AgentBank to get a series of agent models, Samoyed. |
| Outcome: | The proposed model can scale to get generalized agent capabilities. |
SILC-EFSA: Self-aware In-context Learning Correction for Entity-level Financial Sentiment Analysis (2025.coling-main)
Copied to clipboard
Senbin Zhu, ChenYuan He, Hongde Liu, Pengcheng Dong, Hanjie Zhao, Yuchen Yan, Yuxiang Jia, Hongying Zan, Min Peng
| Challenge: | Currently, most sentiment analysis corpora use sequence-level annotation. |
| Approach: | They propose a two-stage approach to financial entity-level sentiment analysis called Self-aware In-context Learning Correction. |
| Outcome: | The proposed approach achieves state-of-the-art on the largest English and Chinese financial entity-level sentiment analysis datasets to date. |
Are Intermediate Layers and Labels Really Necessary? A General Language Model Distillation Method (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge distillation methods rely on intermediate layer features and golden labels, which require aligned model architecture and labeled data respectively. |
| Approach: | They propose a general language model distillation method that performs two-stage word prediction distillation and vocabulary compression, which is simple and shows extremely strong performance. |
| Outcome: | The proposed method outperforms 25 state-of-the-art methods on the SuperGLUE benchmark, achieving an average score that surpasses the best method by 3%. |
Two Heads Are Better Than One: Improving Fake News Video Detection by Correlating with Neighbors (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks for detecting fake news videos are limited . a new approach is proposed to integrate neighborhood information of new videos . |
| Approach: | They propose a framework for automatically detecting fake news videos . it integrates neighborhood relationship of new videos belonging to same event . |
| Outcome: | The proposed framework improves performance of existing detectors and graph aggregation and debunking rectification modules. |
Keyphrase Generation via Soft and Hard Semantic Corrections (2022.emnlp-main)
Copied to clipboard
| Challenge: | Extensive experiments show that CorrKG is capable of generating high-quality keyphrases. |
| Approach: | They propose a correction model CorrKG on top of the MLE pipeline to correct the biases . the adaptive adaptive mass learning scheme is designed to better fit OT and FreqFS . |
| Outcome: | The proposed model overcomes the semantic biases in keyphrase generation using OT and FreqFS techniques. |
UQ-Merge: Uncertainty Guided Multimodal Large Language Model Merging (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing models merging methods often lead to suboptimal performance due to harmful models . et al., 2018; 59: 59-64. |
| Approach: | They propose an uncertainty-guided MLLM merging algorithm that integrates models into a single MLML. |
| Outcome: | The proposed algorithm improves on held-in and held-out vision-language benchmarks. |
Can Large Language Models Be Good Language Teachers? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have achieved remarkable success across diverse domains, but their potential as effective language teachers remains inadequately assessed. |
| Approach: | They propose a framework to evaluate Chinese language teachers' pedagogical competence against international standards. |
| Outcome: | The proposed framework evaluates 13 latest multilingual and Chinese LLMs against international standards for Chinese language teachers. |
Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to generalize deep neural networks are datahungry and generalize poorly from small datasets. |
| Approach: | They propose an agreement score to evaluate routing processes at instance-level and an adaptive optimizer to enhance routing. |
| Outcome: | The proposed approach improves on two NLP tasks and in low-resource settings with few training instances. |
Beyond Ranking: Fine-Grained Diagnostics and Self-Improvement for MLLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Current paradigms rely on holistic scoring and static leaderboards to disentangle fine-grained competencies. |
| Approach: | They propose a framework to shift the focus from ranking to fine-grained diagnosis. |
| Outcome: | The proposed framework surpasses the strongest baseline by 7.92%. |
Efficiency-Effectiveness Reranking FLOPs for LLM-based Rerankers (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing studies evaluate the efficiency of LLM-based rerankers using proxy metrics such as latency and the number of forward passes. |
| Approach: | They propose to use a large language model to evaluate the efficiency of LLM-based rerankers . they propose to measure ranking quality and query processing efficiency using an interpretable FLOPs estimator . |
| Outcome: | The proposed metrics evaluate LLM-based rerankers with different architectures without running any experiments. |
VDebugger: Harnessing Execution Feedback for Debugging Visual Programs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Visual programs are executable code generated by large language models to address visual reasoning problems. |
| Approach: | They propose a critic-refiner framework that localizes and debugs visual programs by tracking execution step by step. |
| Outcome: | The proposed framework detects and corrects program errors leveraging detailed execution feedback, improving interpretability and accuracy. |
DORY: Deliberative Prompt Recovery for LLM (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are used for their groundbreaking performance across various tasks. |
| Approach: | They propose a method that leverages uncertainty to recover prompts accurately using a single LLM without external resources or models. |
| Outcome: | The proposed approach outperforms baselines across diverse LLMs and prompt benchmarks and establishes a new state-of-the-art record in prompt recovery tasks. |
Rumor Detection on Social Media with Crowd Intelligence and ChatGPT-Assisted Networks (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing research on rumor detection challenges the expressive power of text encoding sequences, and insufficient mining of semantic structural information. |
| Approach: | They propose a Crowd Intelligence-based semantic feature learning module to capture textual content’s sequential and hierarchical features and a knowledge-based structural mining module that leverages ChatGPT for knowledge enhancement. |
| Outcome: | The proposed system achieves performance improvement in rumor detection tasks validating the effectiveness and rationality of using large language models as auxiliary tools. |
RepoGenesis: Benchmarking End-to-End Microservice Generation from Readme to Repository (2026.acl-long)
Copied to clipboard
Zhiyuan Peng, Xin Yin, Pu Zhao, Fangkai Yang, Lu Wang, Ran Jia, Xu Chen, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang
| Challenge: | Existing benchmarks focus on isolated function/class-level generation, neglecting complete microservice repository generation. |
| Approach: | They propose a multilingual benchmark for repository-level end-to-end web microservice generation that reflects real-world development workflows. |
| Outcome: | The benchmark compared 106 repositories across 18 domains and 11 frameworks and 1,258 API endpoints and 2,335 test cases. |
Spatial-RAG: Spatial Retrieval Augmented Generation for Real-World Geospatial Reasoning Questions (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing large language models lack spatial computing capabilities and access to up-to-date geospatial data. |
| Approach: | They propose a Retrieval-Augmented Generation framework for geospatial question answering . it integrates structured spatial databases with LLMs via a hybrid spatial retriever . |
| Outcome: | Experiments show that Spatial-RAG significantly improves over baselines. |
Reasoning over Precedents Alongside Statutes: Case-Augmented Deliberative Alignment for LLM Safety (2026.acl-long)
Copied to clipboard
Can Jin, Rui Wu, Tong Che, Qixin Zhang, Hongwu Peng, Jiahui Zhao, Zhenting Wang, Wenqi Wei, Ligong Han, Zhao Zhang, Yuan Cao, Ruixiang Tang, Dimitris N. Metaxas
| Challenge: | OpenAI introduces deliberative alignment (DA) to enhance safety of its o-series models, but effectiveness of this approach in open-source LLMs is understudied. |
| Approach: | They propose a case-augmented deliberative alignment method for large language models . they propose to use reinforcement learning on self-generated safety reasoning chains . |
| Outcome: | The proposed method avoids narrowly enumerated rules and allows broader adaptability. |
Ruleformer: Context-aware Rule Mining over Knowledge Graph (2022.coling-1)
Copied to clipboard
| Challenge: | Existing work on rule mining focuses on mining rules, but how to select appropriate rules for completion of different triplets has not been discussed. |
| Approach: | They propose to take context information into consideration when selecting suitable rules . they devise a transformer-based rule mining approach, Ruleformer . |
| Outcome: | The proposed model takes context information into consideration, which helps select suitable rules for inference tasks. |
Unifying Language Agent Algorithms with Graph-based Orchestration Engine for Reproducible Agent Research (2025.acl-demo)
Copied to clipboard
Qianqian Zhang, Jiajia Liao, Heting Ying, Yibo Ma, Haozhan Shen, Jingcheng Li, Peng Liu, Lu Zhang, Chunxin Fang, Kyusong Lee, Ruochen Xu, Tiancheng Zhao
| Challenge: | Language agents powered by large language models (LLMs) have demonstrated remarkable capabilities in understanding, reasoning, and executing complex tasks. |
| Approach: | They propose a flexible framework that addresses engineering overhead and insufficient evaluation frameworks for fair comparison. |
| Outcome: | The proposed framework simplifies language agent development and establishes a foundation for reproducible agent research. |
FEED PETs: Further Experimentation and Expansion on the Disambiguation of Potentially Euphemistic Terms (2023.starsem-1)
Copied to clipboard
Patrick Lee, Iyanuoluwa Shode, Alain Trujillo, Yuan Zhao, Olumide Ojo, Diana Plancarte, Anna Feldman, Jing Peng
| Challenge: | Existing work on euphemism disambiguation tasks has focused on transformers . euphorias are expressions that soften the message they convey, therefore dictionary-based approaches are ineffective . |
| Approach: | They propose to annotate PETs for vagueness and use transformers to classify PETs . they perform euphemism disambiguation experiments in three different languages . |
| Outcome: | The proposed models perform well in English euphemism disambiguation task . preliminary results will be used to launch future work . |
PROSE: A Pronoun Omission Solution for Chinese-English Spoken Language Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Pro-drop (‘pronoun-dropping’) language requires NMT systems to recover omitted pronouns, but this task lacks sufficient datasets for benchmarking . |
| Approach: | They propose a benchmarking method that leverages the semantic embedding of dropped pronouns to augment training pairs to alleviate the negative impact introduced by pro-drop . |
| Outcome: | The proposed method outperforms existing methods regarding omitted pronoun retrieval and overall translation quality on four Chinese-English translation corpora. |
REFFLY: Melody-Constrained Lyrics Editing Model (2025.naacl-long)
Copied to clipboard
| Challenge: | Automatic melody-to-lyric (M2L) generation aims to create lyrics that align with a given melody. |
| Approach: | They propose a framework for automatic melody-to-lyric generation that allows for a more flexible approach to creating lyrics from plain text. |
| Outcome: | The proposed framework outperforms baselines Lyra and GPT-4 in musicality and text quality. |
Distill The Image to Nowhere: Inversion Knowledge Distillation for Multimodal Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on multimodal machine translation (MMT) have focused on the fusion and alignment of images and texts to improve MMT. |
| Approach: | They propose an image-free inference framework that supports image-based inference via an inversion knowledge distillation scheme. |
| Outcome: | The proposed framework is the first to rival or surpass image-must frameworks on the multimodal translation benchmark. |
Domain-specific Attention with Distributional Signatures for Multi-Domain End-to-end Task-Oriented Dialogue (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to construct multi-domain task-oriented dialogue systems are difficult to extend to new domains due to high cost of data annotation and scarcity of labeled dialogue data. |
| Approach: | They propose a domain attention module that uses distributional signatures to construct multi-domain dialogue systems with limited data. |
| Outcome: | The proposed method outperforms baseline models on most metrics while keeping smaller model scale. |
On Measures of Biases and Harms in NLP (2022.findings-aacl)
Copied to clipboard
Sunipa Dev, Emily Sheng, Jieyu Zhao, Aubrie Amstutz, Jiao Sun, Yu Hou, Mattie Sanseverino, Jiin Kim, Akihiro Nishi, Nanyun Peng, Kai-Wei Chang
| Challenge: | Recent studies show that natural language processing (NLP) technologies propagate societal biases about demographic groups associated with attributes such as gender, race, and nationality. |
| Approach: | They propose a framework for harms and questions to help practitioners understand biases . they propose measurable measures to detect and mitigate biased groups . |
| Outcome: | The proposed framework provides a framework for harms and questions for practitioners to answer to guide the development of bias measures. |
Toward Recognizing More Entity Types in NER: An Efficient Implementation using Only Entity Lexicons (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing named entity recognition systems require large scale labeled data to perform, while annotation of NER data is laborious and time-consuming. |
| Approach: | They propose to adjust an existing named entity recognition system to recognize entity types not defined in the system. |
| Outcome: | The proposed method can be quickly adjusted to a named entity recognition system. |
MEDs for PETs: Multilingual Euphemism Disambiguation for Potentially Euphemistic Terms (2024.findings-eacl)
Copied to clipboard
Patrick Lee, Alain Chirino Trujillo, Diana Cuevas Plancarte, Olumide Ojo, Xinyi Liu, Iyanuoluwa Shode, Yuan Zhao, Anna Feldman, Jing Peng
| Challenge: | Euphemisms are a linguistic device used to soften or neutralize language that may otherwise be harsh or awkward to state directly. |
| Approach: | They train a multilingual transformer model to disambiguate potentially euphemistic terms in multilingual and cross-lingual settings. |
| Outcome: | The proposed model performs better than monolingual models on the disambiguation task compared to monolingual ones in multilingual and cross-lingual settings. |
FinReporting: An Agentic Workflow for Localized Reporting of Cross-Jurisdiction Financial Disclosure (2026.acl-demo)
Copied to clipboard
Fan Zhang, Mingzi Song, Rania Elbadry, Yankai Chen, Shaobo Wang, Yixi Zhou, Xunwen Zheng, Yueru He, Yuyang Dai, Georgi Nenkov Georgiev, Ayesha Gull, Muhammad Usman Safder, Fan Wu, Liyuan Meng, Fengxian Ji, Junning Zhao, Xueqing Peng, Jimin Huang, YU Chen, Xue Liu, Preslav Nakov, Zhuohan Xie
| Challenge: | FinReporting is an agentic workflow for localized cross-jurisdiction financial reporting . existing approaches assume a single-market setting and overlook structural differences across jurisdictions . |
| Approach: | They propose a workflow that decomposes financial reporting into auditable stages . they use Large Language Models to extract and summarize corporate disclosures . |
| Outcome: | The proposed system decomposes reporting into auditable stages . it improves consistency and reliability under heterogeneous reporting regimes. |
Learning Flexible Large Multimodal Models with Arbitrary Modality Combinations (2026.findings-acl)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) have potential for cross-modal understanding . but extending MLLM to handle diverse modalities introduces two challenges . |
| Approach: | They propose a dual-stage compression mechanism to reduce the number of modality tokens per modality and condense it into a single, compact token sequence. |
| Outcome: | Experiments show that Flex-M3 outperforms its counterpart trained on only full-modality data. |
Dialogue is Better Than Monologue: Instructing Meidcal LLMs via Strategic Conversations (2026.findings-eacl)
Copied to clipboard
Zijie Liu, Xinyu Zhao, Jie Peng, Jinhao Duan, Zhuangdi Zhu, Qingyu Chen, Kaidi Xu, Xia Hu, Tianlong Chen
| Challenge: | Existing tuning methods for medical AI models are monologue-based . existing benchmarks are based on licensing exams or research articles . |
| Approach: | They propose a benchmark to expose limitations of monologue-based tuning for medical AI models . they use a large dialogue dataset to capture stepwise diagnostic reasoning . |
| Outcome: | The proposed model outperforms monologue-tuned models on a medical question answering task and improves accuracy on standard medical QA benchmarks. |
LearnDA: Learnable Knowledge-Guided Data Augmentation for Event Causality Identification (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for event causality identification (ECI) rely on annotated training data. |
| Approach: | They propose a method to augment training data for event causality identification by iteratively generating new examples and classifying event causalities in a dual learning framework. |
| Outcome: | The proposed method outperforms existing methods on EventStoryLine and Causal-TimeBank. |
Revisiting the Knowledge Injection Frameworks (2023.emnlp-main)
Copied to clipboard
| Challenge: | Injecting unaligned knowledge tuple into large language models achieves comparable (and sometimes better) results than aligned knowledge. |
| Approach: | They propose a technique to inject random knowledge into large language models to improve performance. |
| Outcome: | The proposed technique overcomes the sanity problem and pushes the performance limit. |
MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application (2026.acl-long)
Copied to clipboard
Xueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang, Yueru He, Yang Ren, Mingyang Jiang, Vincent Jim Zhang, Yuqing Guo, Jeff Zhao, Huan He, Yi Han, Yun Feng, Yuechen Jiang, Yupeng Cao, Haohang Li, Yangyang Yu, Xiaoyu Wang, Penglei Gao, Shengyuan Lin, Keyi Wang, Shanshan Yang, Yilun Zhao, Zhiwei Liu, Peng Lu, Jerry Huang, Suyuchen Wang, Triantafillos Papadopoulos, Polydoros Giannouris, Efstathia Soufleri, Nuo Chen, Zhiyang Deng, Heming Fu, Yijia Zhao, Mingquan Lin, Meikang Qiu, Kaleb E Smith, Arman Cohan, Xiao-Yang Liu, Jimin Huang, Guojun Xiong, Alejandro Lopez-Lira, Xi Chen, Junichi Tsujii, Jian-Yun Nie, Sophia Ananiadou, Qianqian Xie
| Challenge: | Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models. |
| Approach: | They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts. |
| Outcome: | The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings . |
QiMeng-Attention: SOTA Attention Operator is generated by SOTA Attention Algorithm (2025.findings-acl)
Copied to clipboard
Qirui Zhou, Shaohui Peng, Weiqiang Xiong, Haixin Chen, Yuanbo Wen, Haochen Li, Ling Li, Qi Guo, Yongwei Zhao, Ke Gao, Ruizhi Chen, Yanjun Wu, Zhao Chen, Yunji Chen
| Challenge: | Existing LLMs cannot comprehend the complex data flow and computation process of the attention operator and utilize low-level primitive to exploit GPU performance. |
| Approach: | They propose an LLM-friendly Thinking Language (LLM-TL) that can decouple the generation of high-level optimization logic and low-level implementation on GPU and enhance LLMs’ understanding of attention operator. |
| Outcome: | The proposed method outshines existing LLMs on A100, RTX8000, and T4 GPUs, achieving a speed-up of up to 35.16. |
Watch Every Step! LLM Agent Learning via Iterative Step-level Process Refinement (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to enhance agent performance focus on outcome rewards, which may lead to errors or suboptimal actions due to the absence of process supervision signals. |
| Approach: | They propose a step-level framework that provides detailed step-by-step guidance to enhance agent training by using Monte Carlo methods. |
| Outcome: | The proposed framework outperforms strong baselines on three tasks and shows that it is effective in augmenting efficiency and its applicability to diverse models. |
ORCHID: A Chinese Debate Corpus for Target-Independent Stance Detection and Argumentative Dialogue Summarization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent development of large language models (LLMs) have boosted interest on dialogue agents . however, research on these tasks is limited by the insufficiency of public datasets . stance detection and debate summarization are key for engaging argumentative dialogues - but are not available for non-English languages. |
| Approach: | They propose to use ORCHID to benchmark stance detection and debate summarization in Chinese debates. |
| Outcome: | The proposed task is based on 1,218 real-world debates conducted in Chinese on 476 unique topics. |
Cross-lingual Feature Extraction from Monolingual Corpora for Low-resource Unsupervised Bilingual Lexicon Induction (2022.coling-1)
Copied to clipboard
| Challenge: | Unsupervised bilingual lexicon induction models fail on low-resource language pairs due to insufficient initialization. |
| Approach: | They propose a method to learn cross-lingual features from monolingual corpora for low-resource UBLI by integrating cross-linguistic representations with pre-trained word embeddings in a fully unsupervised initialization. |
| Outcome: | The proposed method outperforms state-of-the-art methods on low-resource language pairs and improves representational ability and robustness of existing embedding models. |
Contextualized Semantic Distance between Highly Overlapped Texts (2023.findings-acl)
Copied to clipboard
| Challenge: | Conventional semantic metrics are based on word representations and are vulnerable to disturbance of overlapped components with similar representations. |
| Approach: | They propose a mask-and-predict strategy to evaluate the semantic distance between the overlapped sentences using words in the longest common sequence as neighboring words and use masked language modeling to predict their positions. |
| Outcome: | The proposed method outperforms the state-of-the-art in domain adaption by a huge margin. |
TrigReason: Trigger-Based Collaboration between Small and Large Reasoning Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Reasoning Models suffer from high inference latency due to autoregressive reasoning . SpecReason adopts a polling-based design that repeatedly invokes the LRM for verification at every step . |
| Approach: | They propose a trigger-based collaborative reasoning framework that delegates most reasoning to the SRM and activates LRM intervention only when necessary. |
| Outcome: | The proposed framework reduces latency and API cost by 73.3% under edge–cloud conditions. |
GenWebNovel: A Genre-oriented Corpus of Entities in Chinese Web Novels (2025.coling-main)
Copied to clipboard
| Challenge: | Existing literature on nested entity recognition is insufficient partly due to insufficient annotated data. |
| Approach: | They propose a method that utilizes a pre-trained language model as an In-context learning example retriever to boost the performance of large language models. |
| Outcome: | The proposed method significantly enhances entity recognition, matching state-of-the-art (SOTA) models without additional training data. |
CEAN: Contrastive Event Aggregation Network with LLM-based Augmentation for Event Extraction (2024.eacl-long)
Copied to clipboard
| Challenge: | Event Extraction is a crucial yet arduous task in natural language processing (NLP), as its performance is hindered by laborious data annotation. |
| Approach: | They propose a Contrastive Event Aggregation Network with LLM-based Augmentation to promote low-resource learning and reduce data noise for event extraction. |
| Outcome: | The proposed approach achieves new state-of-the-art results on the ACE2005 and ERE-EN datasets. |
Web Fraud Attacks Against LLM-Driven Multi-Agent Systems (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Model (LLM)-driven multi-agent systems (MAS) are rapidly gaining popularity, and its inherent security risks are rapidly becoming a concern. |
| Approach: | They propose a novel attack manipulating unique structures of web links to deceive MAS by using homoglyph deception, sub-directory nesting, and parameter obfuscation. |
| Outcome: | The proposed attacks exploit unique structures of web links to deceive MAS . they exhibit significant destructive potential across different MAS architectures . |
Meeseeks: A Feedback-Driven, Iterative Self-Correction Benchmark evaluating LLMs’ Instruction Following Capability (2026.findings-acl)
Copied to clipboard
Jiaming Wang, Yunke Zhao, Peng Ding, Jun Kuang, Yibin Shen, Zhe Tang, Yilin Jin, ZongYu Wang, Xiaoyu Li, Xuezhi Cao
| Challenge: | Existing models lack the ability to adhere to instructions, resulting in suboptimal performance. |
| Approach: | They propose an automated iterative instruction-following benchmark with integrated feedback mechanism. |
| Outcome: | The proposed benchmark identifies erroneous components in model responses and provides feedback accurately. |
Table-based Fact Verification with Self-labeled Keypoint Alignment (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for fact verification rely on graph feature or data augmentation but fail to investigate evidence correlation between statement and table effectively. |
| Approach: | They propose a self-labeled keypoint alignment model to explore correlation between statement and table . they propose integrating a mixture-of experts block to integrate interacted information . |
| Outcome: | The proposed model outperforms the state-of-the-art models and captures interpretable evidence words on three widely-studied datasets. |
MPBench: A Comprehensive Multimodal Reasoning Benchmark for Process Errors Identification (2025.findings-acl)
Copied to clipboard
xu Zhao Pan, Pengfei Zhou, Jiaxin Ai, Wangbo Zhao, Kai Wang, Xiaojiang Peng, Wenqi Shao, Hongxun Yao, Kaipeng Zhang
| Challenge: | Existing benchmarks of large language models focus on error detection, neglecting other scenarios like reasoning search. |
| Approach: | et al. propose a multi-task, multimodal benchmark to assess effectiveness of PRMs . step correctness, answers aggregation and reasoning process search are evaluated . ethical principles of MPBench are based on a set of evaluation paradigms based in a text-based benchmark . |
| Outcome: | a new benchmark assesses the effectiveness of large language models (LLMs) in multiple scenarios . it uses three evaluation paradigms to assess the effectiveness and compares them with existing models . a the proposed model improves reasoning accuracy by providing stepwise feedback for multi-step reasoning results . |
DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for reconstruction of large language models overlook diversity among experts, leading to potential redundancy. |
| Approach: | They propose a pruning-based expert reconstruction method that prunes a specific LLM and retrains it on routers, experts and normalization modules. |
| Outcome: | The proposed method outperforms pruning and MoE reconstruction methods on Llama-style models with open-source training corpora. |
CAFE: Retrieval Head-based Coarse-to-Fine Information Seeking to Enhance Multi-Document QA Capability (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to extend context length of Large Language Models (LLMs) still struggle with retrieval and reasoning in long context inputs. |
| Approach: | They propose a coarse-to-fine method to enhance multi-document question-answering capacities by removing background and distracting documents. |
| Outcome: | Experiments show that CAFE outperforms baseline methods on multiple documents. |
FSUIE: A Novel Fuzzy Span Mechanism for Universal Information Extraction (2023.acl-long)
Copied to clipboard
| Challenge: | Existing Universal Information Extraction models rely heavily on span boundaries in data during training, which does not reflect the reality of span annotation challenges. |
| Approach: | They propose a framework that uses fuzzy spans to model various IE tasks . they propose generative Universal Information Extraction (UIE) to unify various ie tasks based on fuzzy span boundaries . |
| Outcome: | The proposed framework improves on a series of main IE tasks with small amounts of data and training epochs. |
Fully Hyperbolic Neural Networks (2022.acl-long)
Copied to clipboard
| Challenge: | Existing hyperbolic neural networks encode features in the hyperbolical space yet formalize most of their operations in the tangent space. |
| Approach: | They propose a fully hyperbolic framework to build hyperbolical networks based on the Lorentz model by adapting Lorentzer transformations to formalize essential operations of neural networks. |
| Outcome: | The proposed framework has better performance on four NLP tasks compared with existing hyperbolic models . |
Rethinking Positional Encoding in Tree Transformer for Code Representation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent works have proposed novel tree Transformers to capture the syntactic structure in source code. |
| Approach: | They propose a novel tree Transformer encoding node positions based on a description method for tree structures to incorporate inductive bias into Transformer. |
| Outcome: | The proposed model outperforms baselines on code summarization and completion tasks across two languages, and it is able to perform better on both local and global paradigms. |
Unleashing the Potentials of Likelihood Composition for Multi-modal Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multi-modal language models with different architectures, parameter sizes, training datasets, and pipelines exhibit varying strengths across different tasks. |
| Approach: | They propose a framework for fusing heterogeneous models off-the-shell, which they call likelihood composition, and introduce basic operations to compose multiple models’ likelihood distribution when doing a multi-choice visual-question-answering task. |
| Outcome: | The proposed framework can be used to fusing heterogeneous models off-the-shell. |
A + B: A General Generator-Reader Framework for Optimizing LLMs to Unleash Synergy Potential (2024.findings-acl)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) is an effective solution to supplement necessary knowledge to large language models. |
| Approach: | They propose a "generate-then-read" pipeline to replace retrieval stage with generation from the LLM itself. |
| Outcome: | The proposed framework outperforms single models in the base and chat versions and addresses safety and helpfulness post-adaptation challenges. |
Actively Supervised Clustering for Open Relation Extraction (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for Open Relation Extraction (OpenRE) use a two-stage pipeline, which learns relation representations and assignments in the first stage, then manually labels relation for each cluster. |
| Approach: | They propose a method that performs relation learning and relation labeling simultaneously without a significant increase in human effort. |
| Outcome: | The proposed method improves existing SOTA methods by 13.8% and 10.6% on two datasets. |
CityNavAgent: Aerial Vision-and-Language Navigation with Hierarchical Semantic Planning and Global Memory (2025.acl-long)
Copied to clipboard
Weichen Zhang, Chen Gao, Shiquan Yu, Ruiying Peng, Baining Zhao, Qian Zhang, Jinqiang Cui, Xinlei Chen, Yong Li
| Challenge: | Existing ground VLN agents struggle in aerial VLLN due to the lack of predefined navigation graphs and the exponentially expanding action space in long-horizon exploration. |
| Approach: | They propose a large language model-empowered aerial VLN agent that decomposes the long-horizon task into sub-goals with different semantic levels. |
| Outcome: | The proposed method achieves state-of-the-art performance with significant improvement in continuous city environments. |
From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for natural language processing (NLP) are fine-tuned and released for research and deployments. |
| Approach: | They propose a PLM reuse paradigm that merges teacher-PLM knowledge into a student model. |
| Outcome: | The proposed paradigm can reduce the computational cost and environmental side-effects of retraining the PLM from scratch. |
BalanceSFT: Improving LLM Function Calling with Balanced Training Signals and Data Hardness (2026.findings-acl)
Copied to clipboard
Bingguang Hao, Zengzhuang Xu, Maolin Wang, Yuntao Wen, Yicheng Chen, Cunyin Peng, Long Chen, Xiangyu Zhao, Jinjie Gu, Chenyi Zhuang, Ji Zhang
| Challenge: | Currently, Supervised Fine-Tuning (SFT) is the prevailing method for equipping Large Language Models (LLMs) with function calling capabilities, but its effectiveness is often compromised by two challenges: 1) lengthy Chain-of-Thought (CoT) reasoning tokens dominate training signals over concise function calls in the learning objective; 2) scarcity of hard training examples. |
| Approach: | They propose a framework that uses a self-adjusted signal balancing loss and a hard data re-sampling strategy to selectively generate new, high-quality complex data guided by model errors. |
| Outcome: | The proposed framework surpasses state-of-the-art models like GPT-5 in function calling performance. |
Knowledge-Enriched Event Causality Identification via Latent Structure Induction Networks (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for identifying causal relations of events are limited . Existing approaches cannot handle well the problem, especially in the condition of lacking training data. |
| Approach: | They propose a Latent Structure Induction Network to integrate external structural knowledge into a causality reasoning task. |
| Outcome: | The proposed approach outperforms existing state-of-the-art methods on two widely used datasets. |
ManCC: A Task-Anchored Benchmark for Manchu–Classical Chinese Cross-Lingual Modeling (2026.findings-acl)
Copied to clipboard
Meiqi Wang, Xiaoxin Sun, Dongjie Wang, Ruixin Yu, Xiantao Heng, Shuo Wang, Zhen Huang, Peng Zhao, Suhua Wang, Minghao Yin
| Challenge: | Mainstream research in natural language processing has focused on high-resource and modern languages. |
| Approach: | They propose a task-anchored benchmark for Manchu–Classical Chinese translation . they use a parallel corpus of 16,627 sentence pairs to evaluate the model . |
| Outcome: | The proposed benchmarks show that linguistic differences influence performance and broader language coverage facilitate low-resource transfer. |
T2R-BENCH: A Benchmark for Real World Table-to-Report Task (2025.emnlp-main)
Copied to clipboard
Jie Zhang, Changzai Pan, Sishi Xiong, Kaiwen Wei, Yu Zhao, Xiangyu Li, Jiaxin Peng, Xiaoyan Gu, Jian Yang, Wenhan Chang, Zhenhe Wu, Jiang Zhong, Shuangyong Song, Xuelong Li
| Challenge: | Existing table benchmarks lack the capacity to adequately assess the practical application of table reasoning in industrial applications. |
| Approach: | They propose a bilingual table-to-report task and a table-based benchmark to assess the quality of table reasoning. |
| Outcome: | The proposed task is based on a bilingual benchmark with 457 industrial tables and evaluation criteria to measure the quality of report generation. |
GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Model (2023.acl-industry)
Copied to clipboard
| Challenge: | Existing knowledge distillation frameworks for language models are limited by memory and the use of complex distillation methods on larger-scale PLMs. |
| Approach: | They propose a general knowledge distillation framework that supports distillation on larger-scale PLMs using various distillation methods. |
| Outcome: | The proposed framework can support distillation on larger-scale PLMs and 25 mainstream methods on 8 NVIDIA A100 (40GB) GPUs. |
Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models enable them to follow freeform instructions, including imitating generic or specific demographic personas in conversations. |
| Approach: | They propose to investigate persona biases by experimenting with UNIVERSALPERSONA, a model that incorporates both generic and specific personas. |
| Outcome: | The proposed model systematically measures persona biases in harmful expression and harmful agreement. |
LLMBox: A Comprehensive Library for Large Language Models (2024.acl-demos)
Copied to clipboard
Tianyi Tang, Hu Yiwen, Bingqian Li, Wenyang Luo, ZiJing Qin, Haoxiang Sun, Jiapeng Wang, Shiyi Xu, Xiaoxue Cheng, Geyang Guo, Han Peng, Bowen Zheng, Yiru Tang, Yingqian Min, Yushuo Chen, Jie Chen, Ranchi Zhao, Luran Ding, Yuhao Wang, Zican Dong, Xia Chunxuan, Junyi Li, Kun Zhou, Xin Zhao, Ji-Rong Wen
| Challenge: | a library to facilitate the development, use, and evaluation of large language models (LLMs) is presented. |
| Approach: | They propose a unified library to facilitate the development, use and evaluation of large language models (LLMs). |
| Outcome: | The proposed library is based on extensive experiments in a variety of evaluation settings. |
BASES: Large-scale Web Search User Simulation with Large Language Model based Agents (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on web search rely on real-user experiments, which can be costly to scale up. |
| Approach: | They propose a user simulation framework with LLM-based agents that can generate unique user profiles at scale. |
| Outcome: | The proposed framework can generate unique user profiles at scale, leading to diverse search behaviors. |
Unsupervised Melody-to-Lyrics Generation (2023.acl-long)
Copied to clipboard
Yufei Tian, Anjali Narayan-Chen, Shereen Oraby, Alessandra Cervone, Gunnar Sigurdsson, Chenyang Tao, Wenbo Zhao, Tagyoung Chung, Jing Huang, Nanyun Peng
| Challenge: | Existing methods for automatic melody-to-lyric generation are limited due to the limited amount of melody-lyrical aligned data. |
| Approach: | They propose a method for automatic melody-to-lyric generation without training on any aligned melody-lyr data. |
| Outcome: | The proposed model generates high-quality lyrics that are singable, intelligible, and coherent than baseline models. |
HSS-Synth: Humanities and Social Sciences Data Synthesis for LLMs (2026.findings-acl)
Copied to clipboard
Ru Peng, Tianyu Zhao, Xijun Gu, Zhiting Fan, Haokai Xu, Jinyang Zhang, Yawen Zeng, Yihong Zhuang, Kexin Yang, Junyang Lin, Dayiheng Liu, Junbo Zhao
| Challenge: | High-quality, diverse data are vital for large language models (LLMs) but remain scarce and costly. |
| Approach: | They define the first HSS domain system covering 14 mainstream fields and introduce HSS-Synth. |
| Outcome: | the proposed pipeline outperforms 14 leading baselines on 16 benchmarks. |
An Electoral Approach to Diversify LLM-based Multi-Agent Collective Decision-Making (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have sparked interest in collaborative LLM agents. |
| Approach: | They propose to integrate various ordinal preferential voting mechanisms into LLMs to improve reasoning capabilities and robustness. |
| Outcome: | The proposed method improves reasoning capabilities and robustness of leading LLMs without complex system designs. |
Stepwise Reasoning Disruption Attack of LLMs (2025.acl-long)
Copied to clipboard
Jingyu Peng, Maolin Wang, Xiangyu Zhao, Kai Zhang, Wanyu Wang, Pengyue Jia, Qidong Liu, Ruocheng Guo, Qi Liu
| Challenge: | Existing attacks on LLM reasoning are constrained by specific settings or lack of imperceptibility, limiting their feasibility and generalizability. |
| Approach: | They propose a stepwise rEasoning error disruption attack that subtly injects errors into prior reasoning steps to mislead the model into producing incorrect subsequent reasoning and final answers. |
| Outcome: | The proposed attack is compatible with zero-shot and few-shot settings, maintains the natural reasoning flow, and ensures covert execution without modification of the instruction. |
A Corpus of Adpositional Supersenses for Mandarin Chinese (2020.lrec-1)
Copied to clipboard
| Challenge: | Adpositions are frequent markers of semantic relations, but they are highly ambiguous and vary significantly from language to language. |
| Approach: | They propose to annotate Chinese adpositions in a corpus with all aforementioned supersenses . they adapt a framework that defined a set of supersens according to ostensibly language-independent criteria . |
| Outcome: | The proposed corpus is the first to be broadly annotated with adposition semantics in Chinese . it shows that the supersense categories are well-suited to Chinese adepositions despite syntactic differences from English . |
M2RC-EVAL: Massively Multilingual Repository-level Code Completion Evaluation (2025.acl-long)
Copied to clipboard
Jiaheng Liu, Ken Deng, Congnan Liu, Jian Yang, Shukai Liu, He Zhu, Peng Zhao, Linzheng Chai, Yanan Wu, JinKe JinKe, Ge Zhang, Zekun Moore Wang, Guoan Zhang, Yingshui Tan, Bangyu Xiang, Zhaoxiang Zhang, Wenbo Su, Bo Zheng
| Challenge: | Existing repository-level code completion benchmarks focus on a limited number of languages . existing benchmarks report overall average scores of different languages ignoring fine-grained abilities . |
| Approach: | They propose to use repository-level code completion benchmarks to evaluate general code intelligence abilities across languages for existing code Large Language Models. |
| Outcome: | The proposed benchmarks improve the code completion abilities of existing LLMs by using two types of annotations on the parsed syntax tree. |
ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | NVFP4 supports fine-grained block isolation, 4-bit quantization errors and mixed-precision approaches . ARCQuant boosts NVFO4 performance via Augmented Residual Channels . |
| Approach: | They propose a framework that boosts NVFP4 performance via Augmented Residual Channels. |
| Outcome: | ARCQuant boosts NVFP4 performance via Augmented Residual Channels . the proposed framework achieves state-of-the-art accuracy comparable to full-precision baselines compared to FP16 . |
FastBERT: a Self-distilling BERT with Adaptive Inference Time (2020.acl-main)
Copied to clipboard
| Challenge: | Pre-trained language models like BERT have proven to be highly performant, but are often computationally expensive in many practical scenarios. |
| Approach: | They propose a speed-tunable FastBERT with adaptive inference time that can be flexibly adjusted under varying demands. |
| Outcome: | The proposed model achieves promising results in English and Chinese datasets. |
Improving Event Causality Identification via Self-Supervised Representation Learning on External Causal Statement (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for event causality identification (ECI) rely on labeled data, but the scale of annotated datasets is limited. |
| Approach: | They propose a self-supervised framework to learn context-specific causal patterns from external causal statements and adopt a contrastive transfer strategy to incorporate the learned context- specific causal patterns into the target ECI model. |
| Outcome: | The proposed method significantly outperforms existing methods on EventSto-ryLine and Causal-TimeBank (+2.0 and +3.4 points on F1 value respectively). |
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)
Copied to clipboard
Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang
| Challenge: | Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction. |
| Approach: | They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack. |
| Outcome: | The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses. |
Embedding and Gradient Say Wrong: A White-Box Method for Hallucination Detection (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for hallucination detection have attracted more attention from the community. |
| Approach: | They propose to model the distributional distance between the regular conditional output and the unconditional output, which is generated without a given input text. |
| Outcome: | The proposed model achieves state-of-the-art on the hallucination benchmarks HADES and other datasets. |
EMO-RL: Emotion-Rule-Based Reinforcement Learning Enhanced Audio-Language Model for Generalized Speech Emotion Recognition (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in reinforcement learning (RL) have shown promise in improving LALMs’ reasoning abilities, but their performance in affective computing tasks remains suboptimal. |
| Approach: | They propose a framework incorporating reinforcement learning with two key innovations: Emotion Similarity-Weighted Reward (ESWR) and Explicit Structured Reasoning (ESR). |
| Outcome: | The proposed framework improves LALMs' reasoning abilities on MELD and IEMOCAP datasets and shows strong generalization. |
Logic Jailbreak: Efficiently Unlocking LLM Safety Restrictions Through Formal Logical Expression (2026.findings-acl)
Copied to clipboard
Jingyu Peng, Maolin Wang, Nan Wang, Jiatong Li, Yuchen Li, Yuyang Ye, Wanyu Wang, Pengyue Jia, Kai Zhang, Xiangyu Zhao
| Challenge: | Despite advances in aligning LLMs with human values, current safety mechanisms remain vulnerable to jailbreak attacks. |
| Approach: | They propose a black-box jailbreak method that uses logical expression translation to bypass LLM safety mechanisms. |
| Outcome: | The proposed method exploits the distributional gap between alignment data and logic-expressed inputs while preserving the underlying semantic intent and readability while evading safety constraints. |
MetaBench: A Multi-task Benchmark for Assessing LLMs in Metabolomics (2026.acl-long)
Copied to clipboard
Yuxing Lu, Xukai Zhao, J. Ben Tamo, Micky C. Nnamdi, Rui Peng, Shuang Zeng, Xingyu Hu, Jinzhuo Wang, May Dongmei Wang
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities on general text, but their proficiency in specialized scientific domains remains uncharacterized. |
| Approach: | They evaluate the capabilities of large language models in metabolomics research using MetaBench . they found that models perform well on text generation tasks, but cross-database identifier grounding remains challenging . |
| Outcome: | The evaluation of 25 open- and closed-source LLMs reveals distinct performance patterns across metabolomics tasks. |
CBP-Tuning: Efficient Local Customization for Black-box Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Customized black-box prompt tuning is a new approach to customize large language models . however, as models grow, the resources required for training and deployment become increasingly expensive . |
| Approach: | They propose a framework that facilitates efficient local customization while preserving bidirectional privacy. |
| Outcome: | The proposed framework facilitates efficient local customization while preserving bidirectional privacy. |