Papers by Wenhao Zhu
Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners (2024.emnlp-main)
Copied to clipboard
Shimao Zhang, Changjiang Gao, Wenhao Zhu, Jiajun Chen, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang
| Challenge: | Large Language Models (LLMs) have shown impressive language capabilities, but most of them have very unbalanced performance across different languages. |
| Approach: | They propose to use question translation data to enhance LLMs' multilingual capabilities by using mechanistic interpretability methods. |
| Outcome: | The proposed method improves multilingual alignment even with unannotated answers in English and a wide range of languages even with instruction-tuned LLMs. |
Learn and Review: Enhancing Continual Named Entity Recognition via Reviewing Synthetic Samples (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for named entity recognition classify mentions into fixed set of predefined entity types but in many real-world scenarios, new entity types are incrementally involved. |
| Approach: | They propose a two-stage framework Learn-and-Review for continual named entity recognition to alleviate inter-type confusion. |
| Outcome: | The proposed framework outperforms the state-of-the-art method on CoNLL-03 and OntoNotes-5.0. |
Retrieval Augmentation for Commonsense Reasoning: A Unified Approach (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for retrieving encyclopedic knowledge lack a large corpus and effective commonsense retriever. |
| Approach: | They propose a framework for retrieval-augmented commonsense reasoning with a large commonsensense corpus and a commonseense retriever. |
| Outcome: | The proposed framework outperforms existing methods on commonsense reasoning tasks. |
STARS: A Unified Framework for Singing Transcription, Alignment, and Refined Style Annotation (2025.findings-acl)
Copied to clipboard
Wenxiang Guo, Yu Zhang, Changhao Pan, Zhiyuan Zhu, Ruiqi Li, ZheTao Chen, Wenhao Xu, Fei Wu, Zhou Zhao
| Challenge: | Existing automated singing annotation (ASA) methods tackle isolated aspects of the annotation pipeline. |
| Approach: | They propose a framework that addresses transcription, alignment, and refined style annotations. |
| Outcome: | The proposed framework delivers comprehensive multi-level annotations encompassing: (1) precise phoneme-audio alignment, (2) robust note transcription and temporal localization, (3) expressive vocal technique identification, and (4) global stylistic characterization including emotion and pace. |
LIFTED: Multimodal Clinical Trial Outcome Prediction via Large Language Models and Mixture-of-Experts (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Clinical trials are costly and pivotal processes that require substantial expenses . a new approach to integrate multimodal data for clinical outcome prediction is needed . |
| Approach: | a proposed framework transforms modality-specific data into natural language descriptions . a sparse Mixture-of-Experts mechanism then identifies shared patterns across modalities . |
| Outcome: | a proposed framework outperforms baseline methods in predicting clinical trial outcomes . it transforms modality-specific data into natural language descriptions, encoded via unified encoders . |
PopAlign: Diversifying Contrasting Patterns for a More Comprehensive Alignment (2025.acl-long)
Copied to clipboard
Zekun Moore Wang, Shenzhi Wang, King Zhu, Jiaheng Liu, Ke Xu, Jie Fu, Wangchunshu Zhou, Wenhao Huang
| Challenge: | Typical approaches to training large language models rely on limited contrasting patterns . contrasting data is limited and models are susceptible to harmful response tendencies . |
| Approach: | They propose a framework that integrates contrasting patterns across the prompt, model, and pipeline levels. |
| Outcome: | The proposed framework outperforms existing methods in the comparison of RQ1 and RQ2 . the proposed framework significantly outperformed existing methods, leading to more comprehensive alignment. |
AgentBank: Towards Generalized LLM Agents via Fine-Tuning on 50000+ Interaction Trajectories (2024.findings-emnlp)
Copied to clipboard
Yifan Song, Weimin Xiong, Xiutian Zhao, Dawei Zhu, Wenhao Wu, Ke Wang, Cheng Li, Wei Peng, Sujian Li
| Challenge: | Existing studies focus on specialized agents designed for particular tasks. |
| Approach: | They propose to scale annotated interaction trajectories and fine-tune LLMs on AgentBank to get a series of agent models, Samoyed. |
| Outcome: | The proposed model can scale to get generalized agent capabilities. |
What Knowledge Is Needed? Towards Explainable Memory for kNN-MT Domain Adaptation (2023.findings-acl)
Copied to clipboard
| Challenge: | kNN-MT builds an external datastore, which saves all target language token occurrences in the parallel corpus. |
| Approach: | They propose a new paradigm for domain adaptation by building an external datastore which usually saves all target language token occurrences in the parallel corpus. |
| Outcome: | The proposed model can be easily pruned according to local correctness, and it is more explainable. |
Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent years have seen a surge of interest in improving the generation quality of commonsense reasoning tasks. |
| Approach: | They propose a method that diversifies the generative reasoning by a mixture of expert strategy on commonsense knowledge graphs to encourage various generation outputs. |
| Outcome: | The proposed method improves diversity while achieving on par performance on two GCR benchmarks, based on both automatic and human evaluations. |
Aligning Large Language Models with Human Preferences through Representation Engineering (2024.acl-long)
Copied to clipboard
Wenhao Liu, Xiaohua Wang, Muling Wu, Tianlong Li, Changze Lv, Zixuan Ling, Zhu JianHao, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Existing methods for achieving this alignment involve employing reinforcement learning from human feedback (RLHF) Existing approaches involve using RLHF to fine-tune LLMs based on human labels . however, RLRF is susceptible to instability during fine- tuning and presents challenges in implementation. |
| Approach: | They propose to use reinforcement learning from human feedback to fine-tune large language models with human preferences to achieve precise control of model behavior. |
| Outcome: | Experiments show that RAHF can be used to capture and manipulate representations to align with a broad spectrum of human preferences or values rather than being confined to a single concept or function. |
Empowering Language Models with Knowledge Graph Reasoning for Open-Domain Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing Language Models lack the power to store all required knowledge, resulting in a lack of ability to infer out-of-context knowledge. |
| Approach: | They propose a Knowledge Interaction Layer that can be flexibly plugged into existing Transformer-based LMs to interact with a differentiable Knowledge Graph Reasoning module collaboratively. |
| Outcome: | The proposed model can be plugged into existing Transformer-based LMs to interact with a differentiable Knowledge Graph Reasoning module collaboratively. |
More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression (2025.findings-emnlp)
Copied to clipboard
Jiebin Zhang, Dawei Zhu, Yifan Song, Wenhao Wu, Chuqiao Kuang, Xiaoguang Li, Lifeng Shang, Qun Liu, Sujian Li
| Challenge: | storing more tokens in the KV cache at lower precision can enhance the long-context performance of large language models. |
| Approach: | They propose a token-precision trade-off strategy to optimize KV cache compression . they also propose storing more tokens in the KV at lower precision . |
| Outcome: | The proposed method achieves an optimal point within the Information Bottleneck compared to standalone KV pruning or KV quantization. |
MAPO: Advancing Multilingual Reasoning through Multilingual-Alignment-as-Preference Optimization (2024.acl-long)
Copied to clipboard
| Challenge: | Existing models exhibit inconsistent reasoning abilities across different languages . existing models lack consistency across languages due to imbalance of training data . |
| Approach: | They propose a multilingual alignment-as-preference optimization framework to align reasoning processes in other languages with the dominant language. |
| Outcome: | The proposed framework improves multilingual reasoning across languages on three benchmarks. |
LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit exceptional translation capabilities in high-resource language tasks, yet their effectiveness in low-resourced languages is suboptimal. |
| Approach: | They conduct extensive multilingual continual pre-training on the LLaMA series models and develop LLiMAX for translation support across more than 100 languages. |
| Outcome: | The proposed model achieves higher translation performance than existing open-source models and performs on-par with specialized translation model on the Flores-101 benchmark. |
APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning (2023.acl-long)
Copied to clipboard
Soumya Sanyal, Yichong Xu, Shuohang Wang, Ziyi Yang, Reid Pryzant, Wenhao Yu, Chenguang Zhu, Xiang Ren
| Challenge: | Existing methods to improve logical reasoning skills require complex data processing. |
| Approach: | They propose an adaptive pretraining approach to improve logical reasoning over text . they use a subset of Wikipedia sentences for pretraining and a sentence-level classification loss . |
| Outcome: | The proposed model outperforms baselines on LogiQA and ReClor. |
DetectBench: Can Large Language Model Detect and Piece Together Implicit Evidence? (2024.findings-emnlp)
Copied to clipboard
Zhouhong Gu, Lin Zhang, Xiaoxuan Zhu, Jiangjie Chen, Wenhao Huang, Yikai Zhang, Shusen Wang, Zheyu Ye, Yan Gao, Hongwei Feng, Yanghua Xiao
| Challenge: | Existing LLMs' abilities to detect evidence in long contexts are far inferior to humans. |
| Approach: | They propose a benchmark to assess LLMs' abilities in evidence and multi-step commonsense reasoning within a long context. |
| Outcome: | The proposed method improves the performance of LLMs in evidence detection and commonsense reasoning. |
Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing monolithic models for multilingual neural machine translation encounter parameter interference and inefficient inference for large models. |
| Approach: | They propose a detachable multi-way model that assigns each language to an individual branch . they use data from OPUS to build a translation benchmark covering 433 languages . |
| Outcome: | The proposed model outperforms existing models in OPUS and is faster than existing models. |
Multilingual Contrastive Decoding via Language-Agnostic Layers Skipping (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Decoding by contrasting layers (DoLa) is designed to improve the generation quality of large language models (LLMs) however, this approach does not work well on non-English tasks. |
| Approach: | They propose a contrastive decoding algorithm that uses amateur logits to contrast with the output of an expert model's early exit logits. |
| Outcome: | The proposed method outperforms baselines and significantly improves chain-of-thought reasoning accuracy across 11 languages. |
FGraDA: A Dataset and Benchmark for Fine-Grained Domain Adaptation in Machine Translation (2022.lrec-1)
Copied to clipboard
Wenhao Zhu, Shujian Huang, Tong Pu, Pingxuan Huang, Xu Zhang, Jian Yu, Wei Chen, Yanfeng Wang, Jiajun Chen
| Challenge: | Recent research on domain adaptation neglects diversity in translation within a domain . current research on NMT models considers very broad target domains . |
| Approach: | They propose a fine-grained domain adaptation task for autonomous vehicles, AI education, real-time networks, and smart phone. |
| Outcome: | The proposed task is compared with a dataset of Chinese-English translation tasks for four sub-domains of information technology: autonomous vehicles, AI education, real-time networks, and smart phone. |
Enhanced Reasoning for Biomedical Document-Level Relation Extraction via a Novel Cascade Language Model Framework (2026.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) are the leading paradigm in document-level relation extraction. |
| Approach: | They propose a cascade framework that leverages the complementary strengths of PLMs and LLMs through a detect-then-rethink paradigm. |
| Outcome: | The proposed framework improves on BioRED and CDR datasets and improves existing models. |
CityCube: Benchmarking Cross-view Spatial Reasoning on Vision-Language Models in Urban Environments (2026.acl-long)
Copied to clipboard
Haotian Xu, Yue Hu, Zhengqiu Zhu, Chen Gao, Ziyou Wang, Junreng Rao, Wenhao Lu, Weishi Li, Quanjun Yin, Yong Li
| Challenge: | Existing benchmarks focus on indoor or street settings, overlooking challenges of open-ended urban spaces. |
| Approach: | They propose a benchmark to probe cross-view spatial reasoning capabilities of current VLMs in urban settings. |
| Outcome: | The citycube benchmark examines the performance of current vision-language models in urban environments. |
kNN-BOX: A Unified Framework for Nearest Neighbor Generation (2024.eacl-demo)
Copied to clipboard
| Challenge: | kNN-BOX enables quick development and visualization for novel generation paradigm . Currently, knn-BOx has provided implementation of seven popular kN-MT variants . |
| Approach: | They propose a framework which decomposes the datastore-augmentation approach into three modules . they apply kNN-BOX to machine translation and three other tasks . |
| Outcome: | The proposed framework decomposes the datastore-augmentation approach into three modules . it provides implementation of seven popular kNN-MT variants, covering research from performance enhancement to efficiency optimization. |
GLTW: Joint Improved Graph Transformer and LLM via Three-Word Language for Knowledge Graph Completion (2025.findings-acl)
Copied to clipboard
Kangyang Luo, Yuzhuo Bai, Cheng Gao, Shuzheng Si, Zhu Liu, Yingli Shen, Zhitong Wang, Cunliang Kong, Wenhao Li, Yufei Huang, Ye Tian, Xuantang Xiong, Lei Han, Maosong Sun
| Challenge: | Existing knowledge graphs lack the ability to integrate structural information into LLMs and output predictions deterministically. |
| Approach: | They propose a method which encodes structural information of KGs and merges it with LLMs to enhance KGC performance. |
| Outcome: | The proposed method improves the performance of KG Completion datasets on KGs by integrating structural information with LLMs. |
ImCoref-CeS: An Improved Lightweight Pipeline for Coreference Resolution with LLM-based Checker-Splitter Refinement (2026.acl-long)
Copied to clipboard
Kangyang Luo, Yuzhuo Bai, Shuzheng Si, Cheng Gao, Zhitong Wang, Yingli Shen, Wenhao Li, Zhu Liu, Yufeng Han, Jiayi Wu, Cunliang Kong, Maosong Sun
| Challenge: | Existing supervised neural methods for coreference resolution are underexplored . current methods rely on small language models, but their potential is underexploited . |
| Approach: | They propose a framework that integrates an enhanced supervised model with LLM-based reasoning. |
| Outcome: | The proposed method surpasses existing state-of-the-art methods in coreference resolution. |
Knowledge-Augmented Methods for Natural Language Processing (2022.acl-tutorials)
Copied to clipboard
| Challenge: | Knowledge in natural language processing (NLP) is a rising trend especially after the advent of large scale pre-trained models. |
| Approach: | This tutorial introduces the key steps in integrating knowledge into natural language processing (NLP) it introduces knowledge grounding from text, knowledge representation and fusing. |
| Outcome: | This tutorial introduces the key steps in integrating knowledge into natural language processing including knowledge grounding from text, knowledge representation and fusing. |
NeuReasoner: Towards Explainable, Controllable, and Unified Reasoning via Mixture-of-Neurons (2026.acl-long)
Copied to clipboard
| Challenge: | Existing Large Reasoning Models (LRMs) lack explainability and controllability . Existing models target isolated levels without unification, while relying on RL . |
| Approach: | They propose an explainable, controllable, and unified reasoning framework driven by MoN. |
| Outcome: | The proposed framework achieves performance gains of 27.0% while reducing token consumption by 19.6% 63.3%. |
MIO: A Foundation Model on Multimodal Tokens (2025.emnlp-main)
Copied to clipboard
Zekun Moore Wang, King Zhu, Chunpu Xu, Wangchunshu Zhou, Jiaheng Liu, Yibo Zhang, Jessie Wang, Ning Shi, Siyu Li, Yizhi Li, Haoran Que, Zhaoxiang Zhang, Yuanxing Zhang, Ge Zhang, Ke Xu, Jie Fu, Wenhao Huang
| Challenge: | Existing models lack multimodal understanding capabilities, resulting in closed-source model that does not support multimodal interleaved sequences. |
| Approach: | They propose a foundation model built on multimodal tokens capable of understanding and generating speech, text, images, and videos in an end-to-end, autoregressive manner. |
| Outcome: | The proposed model is able to understand speech, text, images, and videos in an end-to-end, autoregressive manner. |
INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Neural machine translation models induce a non-smooth representation space, which harms its generalization results. |
| Approach: | They propose a framework to smooth the representation space by adjusting neighbor representations with a small number of new parameters. |
| Outcome: | The proposed framework outperforms the state-of-the-art kNN-MT system with average gains of 1.99 COMET and 1.0 BLEU on four benchmark datasets. |
Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis (2024.findings-naacl)
Copied to clipboard
Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, Lei Li
| Challenge: | Existing studies show that large language models (LLMs) can handle multilingual machine translation (MMT) However, the multilingual translation ability of LLMs remains under-explored. |
| Approach: | They evaluate eight popular LLMs including ChatGPT and GPT-4 to determine their performance in multilingual machine translation. |
| Outcome: | The proposed model can generate moderate translation even on zero-resource languages and cross-lingual exemplars can provide better task guidance for low-resourced translation than exemplar in the same language pairs. |
Sentence-Permuted Paragraph Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models produce similar contents from homogenized contexts due to the fixed left-to-right sentence order. |
| Approach: | They propose a framework permuting sentence orders to improve content diversity of multi-sentence paragraphs by permutating the sentence orders. |
| Outcome: | The proposed framework produces more diverse outputs with higher quality than existing models. |
Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models (2023.findings-emnlp)
Copied to clipboard
Zhihan Zhang, Shuohang Wang, Wenhao Yu, Yichong Xu, Dan Iter, Qingkai Zeng, Yang Liu, Chenguang Zhu, Meng Jiang
| Challenge: | Large language models can perform a wide range of tasks by following natural language instructions without task-specific fine-tuning. |
| Approach: | They propose a method to automatically improve the quality of LLM instructions . they leverage the generative ability of LMS to generate diverse candidate instructions based on a scoring model trained on 575 existing NLP tasks. |
| Outcome: | The proposed method surpasses human-written and LLM-generated instructions on 118 out-of-domain tasks. |
Question Translation Training for Better Multilingual Reasoning (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models have shown compelling performance on reasoning tasks but they tend to perform much worse in languages other than English. |
| Approach: | They propose to train a model to translate reasoning questions into English by fine tuning on X-English parallel question data. |
| Outcome: | The proposed approach improves on LLaMA2-13B on the MGSM and MSVAMP multilingual reasoning benchmarks. |
Dict-BERT: Enhancing Language Model Pre-training with Dictionary (2022.findings-acl)
Copied to clipboard
Wenhao Yu, Chenguang Zhu, Yuwei Fang, Donghan Yu, Shuohang Wang, Yichong Xu, Michael Zeng, Meng Jiang
| Challenge: | Pre-trained language models (PLMs) capture word semantics in different contexts, hence the embeddings of rare words on the tail are poorly optimized. |
| Approach: | They propose to leverage definitions of rare words in dictionaries to enhance language model pre-training by leveraging dictionary definitions. |
| Outcome: | The proposed model improves understanding of rare words and boosts performance on various NLP downstream tasks. |
Advancing Parameter Efficiency in Fine-tuning via Representation Editing (2024.acl-long)
Copied to clipboard
Muling Wu, Wenhao Liu, Xiaohua Wang, Tianlong Li, Changze Lv, Zixuan Ling, Zhu JianHao, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Parameter Efficient Fine-Tuning (PEFT) has gained significant attention for its ability to achieve competitive results while updating only a small subset of trainable parameters. |
| Approach: | They propose a new approach to fine-tuning neural models that scales and biases the representation produced at each layer. |
| Outcome: | The proposed approach reduces the number of trainable parameters by a factor of 25,700 compared to full parameter fine-tuning and by . 32 compared with LoRA. |
SciMMIR: Benchmarking Scientific Multi-modal Information Retrieval (2024.findings-acl)
Copied to clipboard
Siwei Wu, Yizhi Li, Kang Zhu, Ge Zhang, Yiming Liang, Kaijing Ma, Chenghao Xiao, Haoran Zhang, Bohao Yang, Wenhu Chen, Wenhao Huang, Noura Al Moubayed, Jie Fu, Chenghua Lin
| Challenge: | Multi-modal information retrieval (MMIR) is a rapidly evolving field . current benchmarks for image-text pairings overlook the scientific domain . |
| Approach: | They develop a scientific domain-specific MMIR benchmark to evaluate image-text pairings using open-access research paper corpora. |
| Outcome: | The proposed benchmarks are based on 530K image-text pairs extracted from scientific documents with detailed captions. |
MMRA: A Benchmark for Evaluating Multi-Granularity and Multi-Image Relational Association Capabilities in Large Visual Language Models (2026.findings-eacl)
Copied to clipboard
Siwei Wu, King Zhu, Yu Bai, Yiming Liang, Yizhi Li, Haoning Wu, Jiaheng Liu, Ruibo Liu, Xingwei Qu, Xuxin Cheng, Ge Zhang, Wenhao Huang, Chenghua Lin
| Challenge: | Current multimodal benchmarks focus on facts within individual images, but neglect associative relations among multiple images. |
| Approach: | They propose a multi-image relational association task and a MMRA benchmark to evaluate LVLMs. |
| Outcome: | The proposed benchmarks show that entity-level multi-image perception tasks pose greater challenges than image-level tasks. |
A Unified Encoder-Decoder Framework with Entity Memory (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to index, retrieve, and read documents as evidence suffer from large computational overheads. |
| Approach: | They propose an encoder-decoder framework with an entity memory that stores entity knowledge as latent representations and pre-trained on Wikipedia along with encoder parameters. |
| Outcome: | The proposed framework outperforms memory-based and non-memory encoder-decoder models on various entity-intensive question answering and generation tasks. |
Injecting Entity Types into Entity-Guided Text Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in deep generative modeling have led to significant advances in natural language generation (NLG). |
| Approach: | They propose to model the entity type carefully in the decoding phase to generate contextual words accurately. |
| Outcome: | The proposed model produces a target sequence based on a given list of entities. |
Bidirectional LMs are Better Knowledge Memorizers? A Benchmark for Real-world Knowledge Injection (2026.acl-long)
Copied to clipboard
Yuwei Zhang, Wenhao Yu, Shangbin Feng, Yifan Zhu, Letian Peng, Jayanth Srinivasa, Gaowen Liu, Jingbo Shang
| Challenge: | Existing knowledge injection benchmarks for large language models lack standardized testing grounds. |
| Approach: | They propose a knowledge injection benchmark that leverages recently-added and expert-curated facts from Wikipedia’s “Did You Know...” entries. |
| Outcome: | The proposed framework improves reliability accuracy by 29.1%. |
LongEmbed: Extending Embedding Models for Long Context Retrieval (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing embedding models support only 512 input tokens, hindering their application in scenarios requiring long inputs. |
| Approach: | They evaluate the performance of existing embedding models by using a new benchmark and a training-free context window extension strategy. |
| Outcome: | The proposed model extends the input window of existing models by several folds. |
BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multilingual benchmarks focus primarily on language understanding tasks. |
| Approach: | They develop a multi-way multilingual benchmark that measures critical capabilities of large language models across languages. |
| Outcome: | Extensive experiments on BenchMAX reveal uneven utilization of core capabilities across languages, emphasizing the performance gaps that scaling model size alone does not resolve. |
KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering (2022.acl-long)
Copied to clipboard
Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, Michael Zeng
| Challenge: | Open-Domain Question Answering (ODQA) models typically include a retrieving module and a reading module. |
| Approach: | They propose a new open-domain question-answering framework that uses a knowledge-enhanced version of FiD to improve the approach. |
| Outcome: | The proposed model improves on ODQA benchmark datasets with less than 40% computation cost. |
CoUDA: Coherence Evaluation via Unified Data Augmentation (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing data augmentations for coherence evaluation rely on heuristic rules and lack designing criteria. |
| Approach: | They propose a data augmentation framework that breaks down coherence into global and local aspects and designs augmentation strategies for both aspects. |
| Outcome: | The proposed framework surpasses recent models in scoring and ranking tasks with 233M parameters. |
Task Compass: Scaling Multi-task Pre-training with Task Prefix (2022.findings-emnlp)
Copied to clipboard
Zhuosheng Zhang, Shuohang Wang, Yichong Xu, Yuwei Fang, Wenhao Yu, Yang Liu, Hai Zhao, Chenguang Zhu, Michael Zeng
| Challenge: | Existing studies show that multi-task learning with large-scale supervised tasks suffers from negative effects across tasks. |
| Approach: | They propose a task prefix guided multi-task pre-training framework to explore the relationships among tasks. |
| Outcome: | The proposed model can be used as a foundation backbone for a wide range of tasks and as augmentation tool for data augmentation with complementary tasks. |
LIME: Less Is More for MLLM Evaluation (2025.findings-acl)
Copied to clipboard
King Zhu, Qianbo Zang, Shian Jia, Siwei Wu, Feiteng Fang, Yizhi Li, Shuyue Guo, Tianyu Zheng, Jiawei Guo, Bo Li, Haoning Wu, Xingwei Qu, Jian Yang, Ruibo Liu, Xiang Yue, Jiaheng Liu, Chenghua Lin, Hamid Alinejad-Rokny, Min Yang, Shiwen Ni, Wenhao Huang, Ge Zhang
| Challenge: | Existing MLLM benchmarks and unified evaluation frameworks cannot accurately and efficiently reflect the ability of MLMLs. |
| Approach: | They propose a semi-automated benchmark curated using a pipeline that filters out uninformative samples and eliminates answer leakage by focusing on tasks that require image-based understanding. |
| Outcome: | The proposed benchmark reduces the number of samples by 76% and evaluation time by 77% while it can more effectively distinguish different models’ abilities. |
Promoting Data and Model Privacy in Federated Learning through Quantized LoRA (2024.findings-emnlp)
Copied to clipboard
Zhu JianHao, Changze Lv, Xiaohua Wang, Muling Wu, Wenhao Liu, Tianlong Li, Zixuan Ling, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Existing federated learning frameworks require substantial data and computational resources to develop large language models. |
| Approach: | They propose a method that distributes a quantized version of the model’s parameters during training and combine it with a popular fine-tuning method to significantly reduce communication costs. |
| Outcome: | The proposed method enables accurate estimations for parameter updates while preventing clients from accessing a model whose performance is comparable to the centrally hosted one. |
Chain-of-Thought Matters: Improving Long-Context Language Models with Reasoning Path Supervision (2025.findings-emnlp)
Copied to clipboard
Dawei Zhu, Xiyu Wei, Guangxiang Zhao, Wenhao Wu, Haosheng Zou, Junfeng Ran, null XWang, Lin Sun, Xiangzheng Zhang, Sujian Li
| Challenge: | Recent advances in Large Language Models (LLMs) have highlighted the challenge of handling long-context tasks. |
| Approach: | They propose a chain-of-thought framework that teaches models to generate high-quality reasoning paths for enhanced long-context performance. |
| Outcome: | The proposed framework generalizes across most long-context scenarios and amplifys with increasing context length. |