Papers by Yuanbin Wu
Explore Unsupervised Structures in Pretrained Models for Relation Extraction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Syntactic trees are widely used in relation extraction (RE) but they are not stable on different text domains and a pre-defined grammar may not fit the target relation schema. |
| Approach: | They propose to use unsupervised structures to extract relation extraction models . they also conduct detailed analyses on their abilities of adapting new RE domains . |
| Outcome: | The proposed models obtain competitive (even the best) performance scores on benchmark RE datasets. |
Boosting Large Language Models with Continual Learning for Aspect-based Sentiment Analysis (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on improving the performance of domain-specific models based on the target dataset. |
| Approach: | They propose a Large Language Model-based Continual Learning (LLM-CL) model for ABSA that learns the target domain’s ability while maintaining the history domains’ abilities. |
| Outcome: | The proposed model obtains new state-of-the-art over 19 datasets. |
Length Generalization of Causal Transformers without Position Encoding (2024.findings-acl)
Copied to clipboard
| Challenge: | Besides Transformers without position encodings, the success of NoPE provides a new way to overcome the challenge of generalizing to longer sentences. |
| Approach: | They propose a parameter-efficient tuning for searching attention heads’ best temperature hyper-parameters, which substantially expands NoPE’s context size. |
| Outcome: | The proposed tuning significantly expands NoPE's context size, allowing it to generalize to longer sentences with state-of-the-art generalization algorithms. |
Pre-training Entity Relation Encoder with Intra-span and Inter-span Information (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained models do not handle text spans and relation among text span pairs. |
| Approach: | They propose to integrate span-related information into pre-trained encoder for entity relation extraction task. |
| Outcome: | The proposed pre-training method outperforms distantly supervised pre-trained models on two entity relation extraction benchmark datasets. |
Rehearsal-free Continual Language Learning via Efficient Parameter Isolation (2023.acl-long)
Copied to clipboard
Zhicheng Wang, Yufang Liu, Tao Ji, Xiaoling Wang, Yuanbin Wu, Congcong Jiang, Ye Chao, Zhencong Han, Ling Wang, Xu Shao, Wenqiu Zeng
| Challenge: | Existing methods for learning continual tasks do not cache history data, which makes the problem more challenging. |
| Approach: | They propose a method that allocates a small portion of private parameters and learns them with a shared pre-trained model. |
| Outcome: | The proposed method is comparable to existing methods and comparable to those using historical data. |
A Confidence-based Partial Label Learning Model for Crowd-Annotated Named Entity Recognition (2023.findings-acl)
Copied to clipboard
Limao Xiong, Jie Zhou, Qunxi Zhu, Xiao Wang, Yuanbin Wu, Qi Zhang, Tao Gui, Xuanjing Huang, Jin Ma, Ying Shan
| Challenge: | Existing models for named entity recognition (NER) are based on large-scale labeled datasets, which always obtain using crowdsourcing. |
| Approach: | They propose a CONfidence-based partial Label Learning method to integrate prior and posterior confidences for crowd-annotated named entity recognition models. |
| Outcome: | The proposed model improves on real-world and synthetic datasets compared with baselines. |
UniRE: A Unified Label Space for Entity Relation Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing joint entity relation extraction models setup two separate label spaces for the two sub-tasks . |
| Approach: | They propose to eliminate the different treatment on the two sub-tasks’ label spaces by applying a unified classifier to predict each cell’s label. |
| Outcome: | The proposed model achieves competitive accuracy with the best extractor and is faster. |
Word Reordering for Zero-shot Cross-lingual Structured Prediction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current sentence encoders are word order sensitive, resulting in poor performance . Adapting word order from one language to another is key in cross-lingual structured prediction. |
| Approach: | They propose a new module to organize words following the source language order . they build structured prediction models with bag-of-words inputs and introduce a module to do this . |
| Outcome: | The proposed model significantly improves target language performance for languages that are distant from the source language. |
Towards Explainable Chinese Native Learner Essay Fluency Assessment: Dataset, Tasks, and Method (2024.findings-emnlp)
Copied to clipboard
Xinshu Shen, Hongyi Wu, Yadong Zhang, Man Lan, Xiaopeng Bai, Shaoguang Mao, Yuanbin Wu, Xinlin Zhuang, Li Cai
| Challenge: | Existing GEC datasets in Chinese fail to consider specific grammatical error types and overlook cross-sentence grammamatical errors. |
| Approach: | They propose to use Chinese essay fluency assessment to assess essay fluencies along with coarse and fine-grained errors and corrections to improve explainability. |
| Outcome: | The proposed dataset encapsulates essay fluency scores along with both coarse and fine-grained errors and corrections. |
The Role of Visual Modality in Multimodal Mathematical Reasoning: Challenges and Insights (2025.acl-long)
Copied to clipboard
Yufang Liu, Yao Du, Tao Ji, Jianing Wang, Yang Liu, Yuanbin Wu, Aimin Zhou, Mengdi Zhang, Xunliang Cai
| Challenge: | Existing models that leverage visual information do not improve math reasoning performance . authors suggest that visual information is important for multimodal reasoning . |
| Approach: | They propose a dataset to require image reliance for problem-solving and challenge models with similar, yet distinct, images that change the correct answer. |
| Outcome: | The proposed model performance is unaffected by changes to or removal of images in the dataset. |
From Alignment to Assignment: Frustratingly Simple Unsupervised Entity Alignment (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual entity alignment rely on lexical matching and probability reasoning, but they inherit poor interpretability and low efficiency from neural networks. |
| Approach: | They propose a simple but effective unsupervised entity alignment method without neural networks that can be used to find the equivalent entities between crosslingual KGs. |
| Outcome: | Extensive experiments show that the proposed method beats advanced supervised methods across all datasets while having high efficiency, interpretability, and stability. |
Typology Guided Multilingual Position Representations: Case on Dependency Parsing (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent multilingual models benefit from strong unified semantic representation models, but conflicting linguistic regularities may break the effectiveness of word position features in multilingual learning. |
| Approach: | They propose to combine prior knowledge from typology features and existing position vectors to create a position generation network which combines prior knowledge of a language's position space and typological characterization. |
| Outcome: | The proposed model can achieve the best multilingual parsing results by combining prior knowledge from typology features and existing position vectors. |
Graph-based Dependency Parsing with Graph Neural Networks (P19-1)
Copied to clipboard
| Challenge: | In graph-based dependency parsers, learning representations is gaining in importance, and we use graph neural networks to learn the representations. |
| Approach: | They propose to use graph neural networks to learn dependency tree nodes and propose to add a new aggregation function to the system. |
| Outcome: | The proposed model achieves the best UAS and LAS on PTB (96.0%, 94.3%) without using external resources. |
Connective Prediction for Implicit Discourse Relation Recognition via Knowledge Distillation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for implicit discourse relation recognition (IDRR) lack connectives, which is a major challenge in discourse analysis research. |
| Approach: | They propose a method to predict latent correlations between connectives and discourse relations using a knowledge distillation approach. |
| Outcome: | The proposed method outperforms state-of-the-art models on coarse-grained and fine-grain discourse relations and can be transferred to explicit discourse relation recognition and achieve acceptable performance. |
A Unified Encoding of Structures in Transition Systems (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to encode dynamic structures only encode partial information of structures. |
| Approach: | They propose a new attention-based encoder unifying all structures in a transition system. |
| Outcome: | The proposed method significantly improves the test speed and achieves the best transition-based model. |
TASO: Task-Aligned Sparse Optimization for Parameter-Efficient Model Adaptation (2025.emnlp-main)
Copied to clipboard
Daiye Miao, Yufang Liu, Jie Wang, Changzhi Sun, Yunke Zhang, Demei Yan, Shaokang Dong, Qi Zhang, Yuanbin Wu
| Challenge: | Existing studies have shown that LoRA introduces substantial parameter redundancy, which not only increases the number of trainable parameters but also hinders the effectiveness of fine-tuning. |
| Approach: | They propose a method that leverages importance information from the pretrained model’s weights to mitigate LoRA redundancy. |
| Outcome: | The proposed method significantly reduces the number of trainable parameters required for task adaptation while providing a task-aligned perspective for LoRA redundancy reduction. |
SentiX: A Sentiment-Aware Pre-Trained Model for Cross-Domain Sentiment Analysis (2020.coling-main)
Copied to clipboard
| Challenge: | Pre-trained language models have been widely applied to cross-domain NLP tasks like sentiment analysis, but fine-tuning them on the source domain tends to overfit, leading to inferior results on the target domain. |
| Approach: | They propose to pre-train a sentiment-aware language model (SentiX) via domain-invariant sentiment knowledge from large-scale review datasets and utilize it for cross-domain sentiment analysis tasks without fine-tuning. |
| Outcome: | The proposed model achieves state-of-the-art in all the cross-domain sentiment analysis tasks and can be trained with only 1% samples and better than BERT with 90% samples. |
Investigating and Mitigating Object Hallucinations in Pretrained Vision-Language (CLIP) Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have revealed that Large Vision-Language Models suffer from hallucinations in practice, including object hallucines, spatial hallucinos, attribute hallucinications, etc. |
| Approach: | They propose to use CLIP model to mitigate object hallucinations by using a data augmentation method to create negative samples with a variety of hallucinian issues. |
| Outcome: | The proposed method mitigates object hallucinations and can be used as a visual encoder, effectively alleviating the object halluination issue in LVLMs. |
LightEA: A Scalable, Robust, and Interpretable Entity Alignment Framework via Three-view Label Propagation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing EA methods inherit the inborn defects from their neural network lineage: poor interpretability and weak scalability. |
| Approach: | They propose a neural-free EA framework that can find equivalent entity pairs between KGs. |
| Outcome: | The proposed framework has impressive scalability, robustness, and interpretability. |
Probabilistic Graph Reasoning for Natural Proof Generation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to reasoning over formal representations do not explicitly consider inter-dependency between answers and proofs. |
| Approach: | They propose a novel approach for joint answer prediction and proof generation using an induced graphical model. |
| Outcome: | The proposed approach achieves 10%-30% improvement on QA accuracy in evaluations under diverse conditions. |
DVAGen: Dynamic Vocabulary Augmented Generation (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Existing dynamic vocabulary approaches struggle to generalize to novel or out-of-vocabulary words, limiting their flexibility in handling diverse token combinations. |
| Approach: | They propose an open-source framework for training, evaluation, and visualization of dynamic vocabulary-augmented language models. |
| Outcome: | The proposed framework validates the effectiveness of dynamic vocabulary-augmented language models on modern LLMs and shows support for batch inference significantly improving inference throughput. |
A Span-based Linearization for Constituent Trees (2020.acl-main)
Copied to clipboard
| Challenge: | Existing local models have been used to parse constituent trees, but local models can be faster and more efficient. |
| Approach: | They propose a linearization of a constituent tree and a locally normalized model which computes the normalizer on all spans ending with that split point. |
| Outcome: | The proposed model outperforms existing local models and achieves competitive results with global models. |
On Support Samples of Next Word Prediction (2025.acl-long)
Copied to clipboard
| Challenge: | Language models excel in various tasks by making complex decisions, but understanding the rationale behind these decisions remains a challenge. |
| Approach: | They investigate data-centric interpretability in language models by focusing on the next-word prediction task. |
| Outcome: | The proposed model supports or deteres specific predictions, while non-support samples play a critical role in generalization and representation learning. |
CERD: A Comprehensive Chinese Rhetoric Dataset for Rhetorical Understanding and Generation in Essays (2024.findings-emnlp)
Copied to clipboard
Nuowei Liu, Xinhao Chen, Hongyi Wu, Changzhi Sun, Man Lan, Yuanbin Wu, Xiaopeng Bai, Shaoguang Mao, Yan Xia
| Challenge: | Existing rhetorical understanding and generation datasets focus on single coarse-grained categories or fine-grain categories, neglecting the intrinsic connections between different rhetorical devices. |
| Approach: | They propose a Chinese Essay Rhetoric Dataset with four coarse-grained categories . they propose to treat these categories as separate sub-tasks, thereby improving writing skills . |
| Outcome: | The proposed dataset improves the author's writing proficiency and language usage skills by recognizing and generating rhetorical sentences under given conditions. |
Attending via both Fine-tuning and Compressing (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that attention mechanisms can improve models' interpretation, but they are not explicable. |
| Approach: | They propose a framework consisting of a learner and a compressor to purify attention scores . they propose to fine-tune and compress the attention mechanism to obtain a more faithful explanation . |
| Outcome: | The proposed framework improves performance and interpretability on eight benchmark datasets. |
Joint Type Inference on Entities and Relations via Graph Convolutional Networks (P19-1)
Copied to clipboard
| Challenge: | a novel graph convolutional network (GCN) is proposed for the task of joint entity relation extraction. |
| Approach: | They propose a graph convolutional network running on an entity-relation bipartite graph . they propose combining two different methods to perform joint entity relation extraction . |
| Outcome: | The proposed model outperforms existing joint models in entity performance and is competitive with the state-of-the-art in relation performance. |
Few Clean Instances Help Denoising Distant Supervision (2022.coling-1)
Copied to clipboard
| Challenge: | Existing distantly supervised entity relation extractors rely on noisy data for training and evaluation. |
| Approach: | They propose a criterion for clean instance selection based on influence functions to collect sample-level evidence for recognizing good instances. |
| Outcome: | The proposed method shows strong performance on real and synthetic noisy datasets. |
Logic-Regularized Verifier Elicits Reasoning from LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Typical verifiers require resource-intensive supervised dataset construction, which is costly and faces limitations in data diversity. |
| Approach: | They propose an unsupervised verifier regularized by logical rules that uses internal activations and logical constraints on multiple reasoning paths. |
| Outcome: | Experiments on 10 datasets show that the proposed verifier outperforms baselines and is comparable to the supervised verifier. |
Analysing The Impact of Sequence Composition on Language Model Pre-Training (2024.acl-long)
Copied to clipboard
Yu Zhao, Yuanbin Qu, Konrad Staniszewski, Szymon Tworkowski, Wei Liu, Piotr Miłoś, Yuxiang Wu, Pasquale Minervini
| Challenge: | Existing studies show that pretraining sequence composition strategy can lead to distracting information from previous documents. |
| Approach: | They propose to use a sequence construction method to concatenate documents into fixed-length sequences to compute the likelihood of each token given its context. |
| Outcome: | The proposed method can improve in-context learning, knowledge memorisation and context utilisation without sacrificing efficiency. |
Towards Economical Inference: Enabling DeepSeek’s Multi-Head Latent Attention in Any Transformer-based LLMs (2025.acl-long)
Copied to clipboard
Tao Ji, Bin Guo, Yuanbin Wu, Qipeng Guo, Shenlixing Shenlixing, Chenzhan Chenzhan, Xipeng Qiu, Qi Zhang, Tao Gui
| Challenge: | Multi-head Latent Attention (MLA) is an innovative architecture designed to ensure efficient and economical inference by significantly compressing the Key-Value (KV) cache into a latent vector. |
| Approach: | They propose a data-efficient fine-tuning method for transitioning from MHA to MLA using a latent vector cache. |
| Outcome: | The proposed architecture reduces the KV cache size of Llama2-7B by 92.19%, with only 1% drop in LongBench performance. |
Understanding Gender Bias in Knowledge Base Embeddings (2022.acl-long)
Copied to clipboard
| Challenge: | Knowledge base (KB) embeddings have been shown to contain gender biases . authors develop two new bias measures to quantify them and trace their origins in KB . |
| Approach: | They propose two ways to quantify gender biases in knowledge base (KB) embeddings . they use the influence function to inspect the contribution of each triple in KB to the overall group bias . |
| Outcome: | The proposed measures are compared with real-world census data to examine gender biases. |
A Multi-Task Dataset for Assessing Discourse Coherence in Chinese Essays: Structure, Theme, and Logic Analysis (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing research focuses on isolated dimensions of discourse coherence . Existing discourse cohesion analyses focus on isolated aspects of discourse . |
| Approach: | They introduce a Chinese Essay Discourse Coherence Corpus (CEDCC) which integrates coherence grading, topical continuity, and discourse relations. |
| Outcome: | The proposed dataset captures the subtleties of real-world texts and stimulates progress in Chinese discourse coherence analysis. |
Is “hot pizza” Positive or Negative? Mining Target-aware Sentiment Lexicons (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing sentiment lexicons assume words’ sentiments are invariant within a domain, but this assumption is weak for fine-granularity analyses of text sentiments. |
| Approach: | They propose a "perturb-and-see" method to extract commonsense sentiments from large-scale datasets by binding a word's sentiment to its collocation words instead of domain labels. |
| Outcome: | The proposed framework is able to achieve highly competitive performances on the unsupervised opinion relation extraction task. |
Prompt-based Connective Prediction Method for Fine-grained Implicit Discourse Relation Recognition (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to aid implicit discourse relation recognition (IDRR) lack explicit connectives and are difficult to implement on fine-grained IDRR. |
| Approach: | They propose a Prompt-based Connective Prediction method that instructs large-scale pre-trained models to use knowledge relevant to discourse relation and utilizes strong correlation between connectives and discourse relation to help the model recognize implicit discourse relations. |
| Outcome: | The proposed method surpasses the state-of-the-art model and achieves significant improvements on those fine-grained few-shot discourse relation classes. |
Exploring Human Gender Stereotypes with Word Association Test (D19-1)
Copied to clipboard
| Challenge: | Existing word embeddings have been used to study gender stereotypes in texts . however, evaluating their validities is still an open problem . et al.: this study investigates gender bias using the lens of language, especially, the words . |
| Approach: | They use word association test to derive bias scores for large amount of words . they find that these bias scores correlate well with bias in the real world . |
| Outcome: | The proposed method correlates well with bias in the real world, and with census data, it provides a different perspective on gender stereotypes in words. |
Dr3: Ask Large Language Models Not to Give Off-Topic Answers in Open Domain Multi-Hop Question Answering (2024.lrec-main)
Copied to clipboard
Yuan Gao, Yiheng Zhu, Yuanbin Cao, Yinzhi Zhou, Zhen Wu, Yujie Chen, Shenglan Wu, Haoyuan Hu, Xinyu Dai
| Challenge: | Open Domain Multi-Hop Question Answering (ODMHQA) is one of the most challenging tasks in Natural Language Processing (NLP) |
| Approach: | They propose a mechanism that leverages the intrinsic capabilities of Large Language Models to judge whether the generated answers are off-topic. |
| Outcome: | The proposed method reduces the occurrence of off-topic answers by nearly 13%, improving the performance in Exact Match (EM) by nearly 3% compared to the baseline method without the Dr3 mechanism. |
Extracting Entities and Relations with Joint Minimum Risk Training (D18-1)
Copied to clipboard
| Challenge: | Existing methods for detecting entities and relations are limited by the complexity of the joint learning paradigm. |
| Approach: | They propose a joint learning paradigm based on minimum risk training . they implement a strong and simple neural network to execute the MRT . |
| Outcome: | The proposed model is able to achieve state-of-the-art in the extraction task on ACE05 and NYT datasets. |
Generation with Dynamic Vocabulary (2024.emnlp-main)
Copied to clipboard
| Challenge: | Using static vocabulary, vocabulary is ignored in advanced generation tasks. |
| Approach: | They propose a dynamic vocabulary that can involve arbitrary text spans during generation. |
| Outcome: | The proposed vocabulary can be deployed in a plug-and-play way, thus is attractive for various downstream applications. |
CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models pre-trained on massive corpora have shown impressive few-shot learning ability on many NLP tasks. |
| Approach: | They propose to recast structured output in the form of code instead of natural language and use generative LLMs of code to perform IE tasks. |
| Outcome: | The proposed method outperforms fine-tuning moderate-size pre-trained models and prompting NL-LLMs under few-shot settings. |
ENPAR:Enhancing Entity and Entity Pair Representations for Joint Entity Relation Extraction (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for joint entity relation extraction use multitask learning frameworks, but annotations for additional tasks are hard to obtain. |
| Approach: | They propose a pre-training method to improve the joint extraction performance with just extra entity annotations. |
| Outcome: | The proposed method outperforms existing methods on ACE05, SciERC, and NYT and outperformed BERT on other tasks. |