Papers by Wenhao Wu
QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing studies on text summarization factual consistency are divided into two categories . entailment-based and question answering-based metrics are the most efficient . |
| Approach: | They propose an optimized QA-based metric that improves factual consistency by 14% . they compare entailment-based and QA metrics to find the best fit . |
| Outcome: | The proposed metric outperforms the best performing entailment-based metric on the SummaC factual consistency benchmark. |
Controllable Abstractive Dialogue Summarization with Sketch Supervision (2021.findings-acl)
Copied to clipboard
| Challenge: | Using a model to generate summary sketches, we improve abstractive dialogue summarization quality and enable granularity control. |
| Approach: | They propose a model that generates a preliminary summary sketch and a strategy to control granularity. |
| Outcome: | The proposed model achieves state-of-the-art on the largest dialogue summarization corpus with as high as 50.79 in ROUGE-L score. |
Learn and Review: Enhancing Continual Named Entity Recognition via Reviewing Synthetic Samples (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for named entity recognition classify mentions into fixed set of predefined entity types but in many real-world scenarios, new entity types are incrementally involved. |
| Approach: | They propose a two-stage framework Learn-and-Review for continual named entity recognition to alleviate inter-type confusion. |
| Outcome: | The proposed framework outperforms the state-of-the-art method on CoNLL-03 and OntoNotes-5.0. |
Crossing Variational Autoencoders for Answer Retrieval (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods learned semantic representations with dual encoders or dual variational auto-encoders failed to capture the aligned semantics between question and answer. |
| Approach: | They propose to use two variational auto-encoders to generate questions with aligned answers and generating answers with align questions. |
| Outcome: | The proposed method outperforms the state-of-the-art answer retrieval method on SQuAD. |
MixQG: Neural Question Generation with Mixed Answer Types (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing neural question generation approaches focus on short factoid type of answers. |
| Approach: | They propose a neural question generator that trains a single generative model by combining multiple question types with different answer types. |
| Outcome: | The proposed model outperforms existing models in both seen and unseen domains and can generate questions with different cognitive levels when conditioned on different answer types. |
WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing factual consistency metrics are often uncontrollably generating text that is factually inconsistent with inputs. |
| Approach: | They propose a weakly supervised framework that is directly trained on actual generated samples from language models with weakly annotated labels. |
| Outcome: | The proposed framework improves on the TRUE benchmark by 3.3% over existing methods with 435M parameters. |
STARS: A Unified Framework for Singing Transcription, Alignment, and Refined Style Annotation (2025.findings-acl)
Copied to clipboard
Wenxiang Guo, Yu Zhang, Changhao Pan, Zhiyuan Zhu, Ruiqi Li, ZheTao Chen, Wenhao Xu, Fei Wu, Zhou Zhao
| Challenge: | Existing automated singing annotation (ASA) methods tackle isolated aspects of the annotation pipeline. |
| Approach: | They propose a framework that addresses transcription, alignment, and refined style annotations. |
| Outcome: | The proposed framework delivers comprehensive multi-level annotations encompassing: (1) precise phoneme-audio alignment, (2) robust note transcription and temporal localization, (3) expressive vocal technique identification, and (4) global stylistic characterization including emotion and pace. |
AgentBank: Towards Generalized LLM Agents via Fine-Tuning on 50000+ Interaction Trajectories (2024.findings-emnlp)
Copied to clipboard
Yifan Song, Weimin Xiong, Xiutian Zhao, Dawei Zhu, Wenhao Wu, Ke Wang, Cheng Li, Wei Peng, Sujian Li
| Challenge: | Existing studies focus on specialized agents designed for particular tasks. |
| Approach: | They propose to scale annotated interaction trajectories and fine-tune LLMs on AgentBank to get a series of agent models, Samoyed. |
| Outcome: | The proposed model can scale to get generalized agent capabilities. |
DialFact: A Benchmark for Fact-Checking in Dialogue (2022.acl-long)
Copied to clipboard
| Challenge: | Existing fact-checking models trained on non-dialogue data fail to perform well on this task. |
| Approach: | They propose a task of fact-checking in dialogue to improve fact- checking performance . they propose to use an annotated conversational claim and Wikipedia snippets as evidence . |
| Outcome: | The proposed task improves fact-checking performance in dialogue. |
Aligning Large Language Models with Human Preferences through Representation Engineering (2024.acl-long)
Copied to clipboard
Wenhao Liu, Xiaohua Wang, Muling Wu, Tianlong Li, Changze Lv, Zixuan Ling, Zhu JianHao, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Existing methods for achieving this alignment involve employing reinforcement learning from human feedback (RLHF) Existing approaches involve using RLHF to fine-tune LLMs based on human labels . however, RLRF is susceptible to instability during fine- tuning and presents challenges in implementation. |
| Approach: | They propose to use reinforcement learning from human feedback to fine-tune large language models with human preferences to achieve precise control of model behavior. |
| Outcome: | Experiments show that RAHF can be used to capture and manipulate representations to align with a broad spectrum of human preferences or values rather than being confined to a single concept or function. |
Exploring In-Context Learning for Knowledge Grounded Dialog Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge grounded dialog generation models are prone to hallucination and produce factually inaccurate outputs. |
| Approach: | They propose a retrieval-based framework which leverages in-context learning and retrieval techniques to enhance LLMs on knowledge grounded dialog generation. |
| Outcome: | The proposed framework outperforms existing training-based models on a large-scale knowledge graph with 1M+ facts and is expected to perform knowledge-intensive tasks. |
Parameter Efficient Multi-task Fine-tuning by Learning to Transfer Token-wise Prompts (2023.findings-emnlp)
Copied to clipboard
Muling Wu, Wenhao Liu, Jianhan Xu, Changze Lv, Zixuan Ling, Tianlong Li, Longtao Huang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Prompt tuning has been proven to be successful on various tasks by incorporating a small number of trainable parameters while freezing large pre-trained language models. |
| Approach: | They propose a token-wise prompt tuning method that uses a bank of finer-grained soft prompt tokens to generate an instance-dependent prompt. |
| Outcome: | The proposed method performs far better than full parameter fine-tuned models and achieves state-of-the-art by tuning only 0.035% parameters on 14 datasets. |
More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression (2025.findings-emnlp)
Copied to clipboard
Jiebin Zhang, Dawei Zhu, Yifan Song, Wenhao Wu, Chuqiao Kuang, Xiaoguang Li, Lifeng Shang, Qun Liu, Sujian Li
| Challenge: | storing more tokens in the KV cache at lower precision can enhance the long-context performance of large language models. |
| Approach: | They propose a token-precision trade-off strategy to optimize KV cache compression . they also propose storing more tokens in the KV at lower precision . |
| Outcome: | The proposed method achieves an optimal point within the Information Bottleneck compared to standalone KV pruning or KV quantization. |
On-policy Reinforcement Fine-tuning with Offline reward for Multi-step Embodied Planning (2026.acl-long)
Copied to clipboard
| Challenge: | Embodied planning requires agents to make coherent multi-step decisions based on dynamic visual observations and verbal goals. |
| Approach: | They propose an On-policy Reinforcement fine-tuning framework with offline rewards for Embodied Task Planning that preserves generalization benefits of RFT while addressing costly interaction and sparse rewards. |
| Outcome: | The proposed framework outperforms closed-source and online-RL methods on EmbodiedBench, a recent benchmark for interactive embodied tasks. |
Tell Me What You Don’t Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editing (2025.findings-acl)
Copied to clipboard
Wenhao Liu, Siyu An, Junru Lu, Muling Wu, Tianlong Li, Xiaohua Wang, Changze Lv, Xiaoqing Zheng, Di Yin, Xing Sun, Xuanjing Huang
| Challenge: | Role-playing Agents (RPAs) struggle to recognize and respond to hard queries that conflict with their role-play knowledge. |
| Approach: | They propose a lightweight representation editing approach that conveniently shifts conflicting requests to the rejection region, thereby enhancing the model’s refusal accuracy. |
| Outcome: | The proposed model improves RPAs’ refusal ability of conflicting requests while maintaining their general role-playing capabilities. |
FRSUM: Towards Faithful Abstractive Summarization via Enhancing Factual Robustness (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models of abstractive summarization are able to generate fluent and coherent summaries, but they still suffer from the unfaithful generation problem. |
| Approach: | They propose to improve the faithfulness of existing models by enhancing their factual robustness by using a novel training strategy, namely FRSUM, which teaches the model to defend against both explicit adversarial samples and implicit factual adversarials. |
| Outcome: | The proposed training strategy improves faithfulness of various models, such as T5, BART, and T5 . |
MMTE: Corpus and Metrics for Evaluating Machine Translation Quality of Metaphorical Language (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation methods focus on fluency and factual reliability, while neglecting figurative quality. |
| Approach: | They propose a set of human evaluation metrics focused on the translation of figurative language and a parallel metaphor corpus generated by post-editing. |
| Outcome: | The proposed evaluation protocol estimates four aspects of MT: Metaphorical Equivalence, Emotion, Authenticity, and Quality. |
Quiz Design Task: Helping Teachers Create Quizzes with Automated Question Generation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Question generation models are often evaluated with standardized NLG metrics that are based on n-gram overlap. |
| Approach: | They propose to use QGen to help teachers automate the generation of reading comprehension quizzes by comparing n-gram overlap with BLEU to compare system-generated questions with heldout human-written references. |
| Outcome: | The best model had only 68.4% of its questions accepted by the ten teachers who participated in the study. |
Precisely the Point: Adversarial Augmentations for Faithful and Informative Text Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for text generation are weak enough to handle perturbations in inputs, leading to degeneration in faithfulness and informativeness. |
| Approach: | They propose a framework for improving faithfulness and informativeness of Seq2Seq models by perturbing word representations and word swapping. |
| Outcome: | The proposed framework improves faithfulness and informativeness of Seq2Seq models under automatic and human evaluation settings. |
Technical Question Answering across Tasks and Domains (2021.naacl-industry)
Copied to clipboard
| Challenge: | Existing methods for technical QA have a limited data size and question and answer overlaps . |
| Approach: | They propose a framework of deep transfer learning to address technical QA across tasks and domains using document retrieval and reading comprehension tasks. |
| Outcome: | The proposed framework performs better than state-of-the-art methods on the TechQA task. |
CriticLean: Critic-Guided Reinforcement Learning for Mathematical Formalization (2026.acl-long)
Copied to clipboard
Zhongyuan Peng, Yifan Yao, Kaijing Ma, Shuyue Guo, Yizhe Li, Yichi Zhang, Chenchen Zhang, Yifan Zhang, Zhouliang Yu, Luming Li, Minghao Liu, Yihang Xia, Jiawei Shen, Yuchen Wu, Yixin Cao, Zhaoxiang Zhang, Wenhao Huang, Jiaheng Liu, Ge Zhang
| Challenge: | Existing approaches to formalizing mathematical statements face limitations in accuracy, especially in the context of complex, highlevel problems that involve sophisticated mathematical reasoning. |
| Approach: | They propose a CriticLean framework that elevates the role of the critic from a passive validator to an active learning component and introduce a benchmark to measure models’ ability to distinguish semantically correct from incorrect formalizations. |
| Outcome: | The proposed framework outperforms open- and closed-source benchmarks and shows that it significantly outperformed existing models. |
ChatMusician: Understanding and Generating Music Intrinsically with LLM (2024.findings-acl)
Copied to clipboard
Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Liumeng Xue, Ziyang Ma, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Chenghua Lin, Qifeng Liu, Tao Jiang, Wenhao Huang, Wenhu Chen, Jie Fu, Emmanouil Benetos, Gus Xia, Roger Dannenberg, Wei Xue, Shiyin Kang, Yike Guo
| Challenge: | Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task. |
| Approach: | They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5. |
| Outcome: | The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers. |
InstructEval: Instruction-Tuned Text Evaluator from Human Preference (2024.findings-acl)
Copied to clipboard
| Challenge: | InstructEval is a general text evaluator based on open-source Large Language Models (LLMs). |
| Approach: | They propose to build a general multi-aspect text evaluator based on open-source Large Language Models (LLMs) they use extensive open Human Preference Modeling datasets and a small set of multi-spect annotated data to overcome the shortage of annotation resources for multi-task evaluations. |
| Outcome: | The proposed model performs comparable or superior to commercial LLMs like ChatGPT or GPT-4 in terms of both overall and multi-aspect evaluation tasks. |
QSpec: Speculative Decoding with Complementary Quantization Schemes (2025.emnlp-main)
Copied to clipboard
| Challenge: | Quantization is widely adopted to accelerate inference and reduce memory consumption in large language models. |
| Approach: | They propose a quantization paradigm that decouples efficiency from quality by integrating two complementary schemes via speculative decoding. |
| Outcome: | The proposed approach achieves 1.64x speedup without quality degradation and outperforms state-of-the-art speculative decoding methods by 1.55x in batched settings. |
Conformal Predictor for Improving Zero-Shot Text Classification Efficiency (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) have been shown effective for zero-shot (0shot) text classification. |
| Approach: | They propose to limit the number of likely labels using a fast base classifier-based conformal predictor calibrated on samples labeled by the 0shot model. |
| Outcome: | The proposed models reduce the average inference time for NLI- and NSP-based models by 25.6% and 22.2% without dropping performance below the predefined error rate of 1%. |
Music Audio-Visual Question Answering Requires Specialized Multimodal Designs (2026.findings-acl)
Copied to clipboard
Wenhao You, Xingjian Diao, Wenjun Huang, Chunhui Zhang, Keyi Kong, Weiyi Wu, Chiyu Ma, Zhongyu Ouyang, Tingxuan Wu, Ming Cheng, Soroush Vosoughi, Jiang Gui
| Challenge: | Music audio-visual question answering presents unique challenges with dense audio-visual content, intricate temporal dynamics, and the need for domain-specific knowledge. |
| Approach: | They analyze Music AVQA datasets and analyze their results to identify key design patterns . they propose concrete future directions for incorporating musical priors . |
| Outcome: | The proposed architectures are critical for success in Music AVQA, the authors argue . they suggest concrete future directions for incorporating musical priors . |
QAConv: Question Answering on Informative Conversations (2022.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that state-of-the-art pretrained QA systems have limited zero-shot performance and tend to predict our questions as unanswerable. |
| Approach: | They propose a question-answering dataset that uses conversations as a knowledge source. |
| Outcome: | The proposed dataset provides a training and evaluation testbed to facilitate QA on conversations research. |
ImCoref-CeS: An Improved Lightweight Pipeline for Coreference Resolution with LLM-based Checker-Splitter Refinement (2026.acl-long)
Copied to clipboard
Kangyang Luo, Yuzhuo Bai, Shuzheng Si, Cheng Gao, Zhitong Wang, Yingli Shen, Wenhao Li, Zhu Liu, Yufeng Han, Jiayi Wu, Cunliang Kong, Maosong Sun
| Challenge: | Existing supervised neural methods for coreference resolution are underexplored . current methods rely on small language models, but their potential is underexploited . |
| Approach: | They propose a framework that integrates an enhanced supervised model with LLM-based reasoning. |
| Outcome: | The proposed method surpasses existing state-of-the-art methods in coreference resolution. |
From Scaffolding to Assimilation: Progressive Structural Internalization for Format-Constrained Creative Text Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing paradigms rely on unreliable prompting or rigid constrained decoding strategies to achieve aesthetic unity. |
| Approach: | They propose a framework to embed external constraints into the model’s intrinsic intuition and use it to generate open-ended creative texts. |
| Outcome: | The proposed framework surpasses baselines in both strict constraint adherence and literary aesthetics. |
Debiasing Generative Named Entity Recognition by Calibrating Sequence Likelihood (2023.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to recognize flat, overlapped and discontinuous entities uniformly have been used for Named Entity Recognition. |
| Approach: | They propose a reranking-based approach that redistributes the likelihood among candidate sequences depending on their performance via a contrastive loss. |
| Outcome: | The proposed method boosts baseline and yields competitive or better results compared with the state-of-the-art methods on 8 widely-used datasets for Named Entity Recognition. |
Watch Every Step! LLM Agent Learning via Iterative Step-level Process Refinement (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to enhance agent performance focus on outcome rewards, which may lead to errors or suboptimal actions due to the absence of process supervision signals. |
| Approach: | They propose a step-level framework that provides detailed step-by-step guidance to enhance agent training by using Monte Carlo methods. |
| Outcome: | The proposed framework outperforms strong baselines on three tasks and shows that it is effective in augmenting efficiency and its applicability to diverse models. |
Exploring Neural Models for Query-Focused Summarization (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent work in Query-focused summarization lacks a comprehensive study of the broad space of applicable modeling methods. |
| Approach: | They propose to explore two general classes of methods for Query-focused summarization: extractive-abstractive solutions and end-to-end models. |
| Outcome: | The proposed models achieve state-of-the-art on the QMSum dataset, with a margin of 3.38 ROUGE-1, 3.72 ROUGe2 and 3.28 ROUGEL-L. |
UPLex: Fine-Grained Personality Control in Large Language Models via Unsupervised Lexical Modulation (2025.findings-emnlp)
Copied to clipboard
Tianlong Li, Wenhao Liu, Muling Wu, Shihan Dou, Zhenghua Wang, Changze Lv, Xiaohua Wang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Personality is a crucial factor that shapes human communication patterns, thereby regulating the personalities of large language models (LLMs). |
| Approach: | They propose a method that uses an Unsupervisedly-Built Personalized Lexicon (UPL) during the decoding phase to manipulate LLM’s personality traits. |
| Outcome: | The proposed method can modulate the personality expression of large language models by dynamically altering their predicted probability of upcoming words in a pluggable fashion. |
Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference (2020.emnlp-main)
Copied to clipboard
Jianguo Zhang, Kazuma Hashimoto, Wenhao Liu, Chien-Sheng Wu, Yao Wan, Philip Yu, Richard Socher, Caiming Xiong
| Challenge: | Existing work on few-shot intent classification without OOS has focused on the few-shot intent classification with out-of-scope intents. |
| Approach: | They propose to use BERT-style pairwise encoding to train a binary classifier that estimates the best matched training example for a user input. |
| Outcome: | The proposed approach achieves more stable and accurate in-domain and OOS detection accuracy than RoBERTa-based classifiers and embedding-based nearest neighbor approaches. |
COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values (2026.findings-eacl)
Copied to clipboard
Siwei Wu, JinCheng Ren, Xeron Du, Shuyue Guo, Xingwei Qu, Yiming Liang, Jie Liu, Yunwen Li, Tyler Loakman, Tianyu Zheng, Boyu Feng, Huaqing Yuan, Zili Wang, Jiaheng Liu, Wenhao Huang, Chenglin Cai, Haoran Que, Jian Yang, Yuelin Bai, Zekun Moore Wang, Zhouliang Yu, Qunshu Lin, Ding Pan, Yuchen Eleanor Jiang, Tiannan Wang, Wangchunshu Zhou, Shenzhi Wang, Xingyuan Bu, Minghao Liu, Guoyin Wang, Ge Zhang, Chenghua Lin
| Challenge: | Existing Chinese preference datasets suffer from limited scale, restricted domain coverage, and insufficiently rigorous data validation. |
| Approach: | They propose an LLM-based data annotation pipeline with no human intervention to annotate Chinese preference datasets. |
| Outcome: | The proposed pipeline outperforms existing Chinese preference datasets on AlignBench and Chinese Reward Benchmark. |
Advancing Parameter Efficiency in Fine-tuning via Representation Editing (2024.acl-long)
Copied to clipboard
Muling Wu, Wenhao Liu, Xiaohua Wang, Tianlong Li, Changze Lv, Zixuan Ling, Zhu JianHao, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Parameter Efficient Fine-Tuning (PEFT) has gained significant attention for its ability to achieve competitive results while updating only a small subset of trainable parameters. |
| Approach: | They propose a new approach to fine-tuning neural models that scales and biases the representation produced at each layer. |
| Outcome: | The proposed approach reduces the number of trainable parameters by a factor of 25,700 compared to full parameter fine-tuning and by . 32 compared with LoRA. |
SciMMIR: Benchmarking Scientific Multi-modal Information Retrieval (2024.findings-acl)
Copied to clipboard
Siwei Wu, Yizhi Li, Kang Zhu, Ge Zhang, Yiming Liang, Kaijing Ma, Chenghao Xiao, Haoran Zhang, Bohao Yang, Wenhu Chen, Wenhao Huang, Noura Al Moubayed, Jie Fu, Chenghua Lin
| Challenge: | Multi-modal information retrieval (MMIR) is a rapidly evolving field . current benchmarks for image-text pairings overlook the scientific domain . |
| Approach: | They develop a scientific domain-specific MMIR benchmark to evaluate image-text pairings using open-access research paper corpora. |
| Outcome: | The proposed benchmarks are based on 530K image-text pairs extracted from scientific documents with detailed captions. |
RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models (2024.findings-acl)
Copied to clipboard
Noah Wang, Z.y. Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhao Huang, Jie Fu, Junran Peng
| Challenge: | Large Language Models (LLMs) have paved the way for complex tasks such as role-playing. |
| Approach: | They propose a framework to benchmark, elicit, and enhance role-playing abilities in Large Language Models. |
| Outcome: | The proposed framework improves role-playing abilities with 168,093 samples. |
BASS: Boosting Abstractive Summarization with Unified Semantic Graph (2021.acl-long)
Copied to clipboard
| Challenge: | Abstractive summarization for long-document or multi-document remains challenging for Seq2Seq as it does not analyze long-distance relations in text. |
| Approach: | They propose a framework for Boosting Abstractive Summarization based on a unified Semantic graph which aggregates co-referent phrases distributing across a long range of context and conveys rich relations between phrases. |
| Outcome: | The proposed framework improves document representation and summary generation process by leveraging the graph structure. |
MMRA: A Benchmark for Evaluating Multi-Granularity and Multi-Image Relational Association Capabilities in Large Visual Language Models (2026.findings-eacl)
Copied to clipboard
Siwei Wu, King Zhu, Yu Bai, Yiming Liang, Yizhi Li, Haoning Wu, Jiaheng Liu, Ruibo Liu, Xingwei Qu, Xuxin Cheng, Ge Zhang, Wenhao Huang, Chenghua Lin
| Challenge: | Current multimodal benchmarks focus on facts within individual images, but neglect associative relations among multiple images. |
| Approach: | They propose a multi-image relational association task and a MMRA benchmark to evaluate LVLMs. |
| Outcome: | The proposed benchmarks show that entity-level multi-image perception tasks pose greater challenges than image-level tasks. |
Trust Within? Seek Beyond? Knowledge Boundary Aware Policy Optimization for Agentic Search (2026.acl-long)
Copied to clipboard
Tao Feng, Xinke Jiang, Xinyan Hu, Yonggang Zhang, Zhen Tao, Wentao Zhang, Boyang Liu, Wenhao Jiang, Chao Wu
| Challenge: | Existing approaches to augment large language models with external knowledge suffer from a lack of calibration regarding the model’s knowledge boundary. |
| Approach: | They propose a reinforcement learning framework that explicitly aligns retrieval decisions with quantified knowledge states. |
| Outcome: | The proposed framework outperforms strong baselines while exhibiting reduced hallucination rates. |
Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective (2025.coling-main)
Copied to clipboard
Tianlong Li, Zhenghua Wang, Wenhao Liu, Muling Wu, Shihan Dou, Changze Lv, Xiaohua Wang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Recent surge in jailbreaking attacks has revealed significant vulnerabilities in Large Language Models (LLMs) however, limited research into the underlying mechanisms that make LLMs vulnerable to such attacks has been conducted. |
| Approach: | They propose that LLMs' self-safeguarding capability is linked to specific activity patterns within their representation space. |
| Outcome: | The proposed models can be detected with a few pairs of contrastive queries, and the robustness can be manipulated by weakening or strengthening these patterns. |
Composing Elementary Discourse Units in Abstractive Summarization (2020.acl-main)
Copied to clipboard
| Challenge: | Abstractive summarization uses a single document sentence to generate a summary, but this can cause performance degradation. |
| Approach: | They propose to use elementary discourse unit (EDU) as the summarization unit to extract and group informative EDUs and then an EDU fusion model to fuse the EDU in each group into one sentence. |
| Outcome: | The proposed model can be used to combine informative EDUs into one sentence and reward selection actions. |
LongEmbed: Extending Embedding Models for Long Context Retrieval (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing embedding models support only 512 input tokens, hindering their application in scenarios requiring long inputs. |
| Approach: | They evaluate the performance of existing embedding models by using a new benchmark and a training-free context window extension strategy. |
| Outcome: | The proposed model extends the input window of existing models by several folds. |
A Technical Question Answering System with Transfer Learning (2020.emnlp-demos)
Copied to clipboard
| Challenge: | StackOverflow and AskUbuntu are popular open forum communities for technical question-answering, but it is expensive for human experts to provide timely and helpful responses. |
| Approach: | They develop a system that automatically responds to questions from a siamese ALBERT network based on previously answered questions . |
| Outcome: | The proposed system responds automatically to questions based on answers from previous users. |
CaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work suggests that the degree of hallucination depends on factual errors in training data. |
| Approach: | They propose a method to use training data to reduce hallucination by ensembling parameter variations in training data. |
| Outcome: | The proposed method improves on XSUM and CNN/DM datasets on human evaluations and factual metrics. |
COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning (2025.findings-naacl)
Copied to clipboard
Yuelin Bai, Xeron Du, Yiming Liang, Leo Jin, Junting Zhou, Ziqiang Liu, Feiteng Fang, Mingshan Chang, Tianyu Zheng, Xincheng Zhang, Nuo Ma, Zekun Moore Wang, Ruibin Yuan, Haihong Wu, Hongquan Lin, Wenhao Huang, Jiajun Zhang, Chenghua Lin, Jie Fu, Min Yang, Shiwen Ni, Ge Zhang
| Challenge: | Existing datasets for Chinese instruction tuning are not well-aligned with Chinese users’ interaction patterns. |
| Approach: | They propose to use Chinese instruction tuning datasets to improve instruction fine-tuning for Chinese users. |
| Outcome: | The proposed dataset shows that Chinese models achieve competitive performance in diverse benchmarks. |
CoUDA: Coherence Evaluation via Unified Data Augmentation (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing data augmentations for coherence evaluation rely on heuristic rules and lack designing criteria. |
| Approach: | They propose a data augmentation framework that breaks down coherence into global and local aspects and designs augmentation strategies for both aspects. |
| Outcome: | The proposed framework surpasses recent models in scoring and ranking tasks with 233M parameters. |
LIME: Less Is More for MLLM Evaluation (2025.findings-acl)
Copied to clipboard
King Zhu, Qianbo Zang, Shian Jia, Siwei Wu, Feiteng Fang, Yizhi Li, Shuyue Guo, Tianyu Zheng, Jiawei Guo, Bo Li, Haoning Wu, Xingwei Qu, Jian Yang, Ruibo Liu, Xiang Yue, Jiaheng Liu, Chenghua Lin, Hamid Alinejad-Rokny, Min Yang, Shiwen Ni, Wenhao Huang, Ge Zhang
| Challenge: | Existing MLLM benchmarks and unified evaluation frameworks cannot accurately and efficiently reflect the ability of MLMLs. |
| Approach: | They propose a semi-automated benchmark curated using a pipeline that filters out uninformative samples and eliminates answer leakage by focusing on tasks that require image-based understanding. |
| Outcome: | The proposed benchmark reduces the number of samples by 76% and evaluation time by 77% while it can more effectively distinguish different models’ abilities. |
Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for evaluating progress in natural language generation tasks are expensive, difficult to reproduce, and non-reusable. |
| Approach: | They propose a new automatic evaluation method for NLG called Near-Negative Distinction that repurposes prior human annotations into NND tests. |
| Outcome: | The proposed method achieves higher correlation with human judgments than standard NLG evaluation metrics. |
Promoting Data and Model Privacy in Federated Learning through Quantized LoRA (2024.findings-emnlp)
Copied to clipboard
Zhu JianHao, Changze Lv, Xiaohua Wang, Muling Wu, Wenhao Liu, Tianlong Li, Zixuan Ling, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Existing federated learning frameworks require substantial data and computational resources to develop large language models. |
| Approach: | They propose a method that distributes a quantized version of the model’s parameters during training and combine it with a popular fine-tuning method to significantly reduce communication costs. |
| Outcome: | The proposed method enables accurate estimations for parameter updates while preventing clients from accessing a model whose performance is comparable to the centrally hosted one. |
Chain-of-Thought Matters: Improving Long-Context Language Models with Reasoning Path Supervision (2025.findings-emnlp)
Copied to clipboard
Dawei Zhu, Xiyu Wei, Guangxiang Zhao, Wenhao Wu, Haosheng Zou, Junfeng Ran, null XWang, Lin Sun, Xiangzheng Zhang, Sujian Li
| Challenge: | Recent advances in Large Language Models (LLMs) have highlighted the challenge of handling long-context tasks. |
| Approach: | They propose a chain-of-thought framework that teaches models to generate high-quality reasoning paths for enhanced long-context performance. |
| Outcome: | The proposed framework generalizes across most long-context scenarios and amplifys with increasing context length. |