Papers by Jianfei Yu
Grounded Multimodal Named Entity Recognition on Social Media (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies on Multimodal Named Entity Recognition only extract entity-type pairs in text, which is useless for multimodal knowledge graph construction. |
| Approach: | They propose a task to identify named entities in text and their bounding box groundings in image . they extend four well-known MNER methods to establish a number of baseline systems . |
| Outcome: | The proposed framework outperforms baseline systems on the GMNER task. |
Flexible Thinking for Multimodal Emotional Support Conversation via Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current Chain-of-Thought based ESC methods often employ rigid, text-only reasoning, limiting adaptability in dynamic, multimodal interactions and introducing reasoning noise that degrades support quality. |
| Approach: | They propose a framework that integrates supervised fine-tuning with reinforcement learning to improve ESC models' response quality. |
| Outcome: | The proposed framework enables models to select contextually relevant thinking aspects: Visual Scene, Emotion, Situation, and Response Strategy. |
Multilingual Synopses of Movie Narratives: A Dataset for Vision-Language Story Understanding (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Story video-text alignment is a core task in computational story understanding, but its progress has been held back by the scarcity of manually annotated video- text correspondences and the heavy concentration on English narrations of Hollywood movies. |
| Approach: | They construct a multilingual video story dataset with 13,166 movie summary videos from 7 languages and manual annotations of fine-grained video-text correspondences. |
| Outcome: | The proposed approach outperforms the SOTA methods on clip accuracy and Sentence IoU scores. |
Cross-Domain Review Generation for Aspect-Based Sentiment Analysis (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing domain adaptation methods for Aspect-Based Sentiment Analysis lack finegrained labeled data. |
| Approach: | They propose a new domain adaptation paradigm called cross-domain review generation which aims to generate target-domain reviews with fine-grained annotation based on the labeled source domain. |
| Outcome: | The proposed approach is superior to state-of-the-art domain adaptation methods. |
Interactive Semantic Parsing with Reinforcement Learning for Knowledge Graph Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to improve LLM reliability rely on factual hallucinations . Existing methods rely only on graph traversal, resulting in imprecise retrieval and heavy post-processing burdens. |
| Approach: | They propose a framework that integrates knowledge Graphs as structured, high-fidelity buffers to enhance LLM reliability. |
| Outcome: | The proposed framework allows logical constraints to be dynamically interleaved with graph search while optimizing via reinforcement learning with only final answer feedback eliminates the need for gold program annotations. |
A Sequence-to-Structure Approach to Document-level Targeted Sentiment Analysis (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis (ABSA) has received wide attention in NLP for nearly two decades . previous studies focused on sentence-level ABSA, but document-level research has not received enough attention. |
| Approach: | They propose a Sequence-to-Structure approach to address the document-level targeted sentiment analysis task, which aims to extract the opinion targets consisting of multi-level entities from a review document and predict their sentiments. |
| Outcome: | The proposed approach outperforms baselines on six domains on the document-level targeted sentiment analysis task. |
UniCOQE: Unified Comparative Opinion Quintuple Extraction As A Set (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods decompose the COQE task into multiple subtasks and solve them in a pipeline manner, but ignore the intrinsic connection between subtask and the error propagation among stages. |
| Approach: | They propose a unified generative model that solves COQE in one shot by concatenating all the comparative tuples into a target output sequence. |
| Outcome: | The proposed model significantly outperforms the SOTA method on multiple benchmarks and ablation experiments. |
ECPE-2D: Emotion-Cause Pair Extraction based on Joint Two-Dimensional Representation, Interaction and Prediction (2020.acl-main)
Copied to clipboard
| Challenge: | a new task, called emotion-cause pair extraction, has emerged in text emotion analysis . a 2D representation scheme is proposed to represent the emotion-case pairs . |
| Approach: | They propose a 2D approach to represent emotion-cause pairs by a 3D representation scheme. |
| Outcome: | The proposed approach improves the state-of-the-art on the emotion cause corpus . the proposed approach is based on a two-step framework with flaws . |
Aspect-Category based Sentiment Analysis with Hierarchical Graph Convolutional Network (2020.coling-main)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis studies focus on identifying sentiment polarities toward explicit aspects but ignore implicit aspects in text. |
| Approach: | They propose a hierarchy-sentiment hierarchy prediction problem to capture explicit and implicit aspects of aspect-based sentiment analysis. |
| Outcome: | The proposed model can consistently achieve the best results on four benchmarks. |
Emotion Cause Extraction on Social Media without Human Annotation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have focused on extracting emotion causes from news articles, but lack of fine-grained annotations has limited the ECE task. |
| Approach: | They propose a new ECE framework that extracts emotion causes from social media data without relying on human annotations. |
| Outcome: | The proposed framework achieves high extraction performance and generalizability without relying on human annotations. |
Improving Multi-label Emotion Classification via Sentiment Classification with Dual Attention Transfer Network (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to emotion detection are lexicon-based, graphical model-based and linear classifier-based. |
| Approach: | They propose a transfer learning architecture to divide sentence representation into two different feature spaces which capture general sentiment words and other important emotion-specific words via a dual attention mechanism. |
| Outcome: | The proposed model can capture general sentiment words and other emotion-specific words via a dual attention mechanism on two benchmark datasets. |
Generative Cross-Domain Data Augmentation for Aspect and Opinion Co-Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to perform aspect and opinion co-extraction are difficult due to the lack of fine-grained annotations. |
| Approach: | They propose a framework to transfer knowledge from a labeled source domain to an unlabeled target domain. |
| Outcome: | The proposed framework is more effective than previous domain adaptation methods on three datasets. |
A Facial Expression-Aware Multimodal Multi-task Learning Framework for Emotion Recognition in Multi-party Conversations (2023.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown the importance of visual information in multi-party conversations due to the complexity of visual scenes. |
| Approach: | They propose a framework to extract face sequences as visual features from a real speaker's utterance and a pipeline method to extract the face sequence. |
| Outcome: | The proposed framework extracts face sequences of the real speaker of each utterance and improves emotion prediction on the MELD dataset. |
Reinforced Counterfactual Data Augmentation for Dual Sentiment Classification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve generalization ability by augmenting training data with synonymous examples or adding random noises to word embeddings cannot address spurious association problem. |
| Approach: | They propose an end-to-end reinforcement learning framework which jointly performs counterfactual data generation and dual sentiment classification. |
| Outcome: | The proposed framework outperforms strong data augmentation baselines on several benchmark sentiment classification datasets. |
A State-independent and Time-evolving Network for Early Rumor Detection in Social Media (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to rumor detection ignored dynamical evolution of an event and failed to capture its unique features in different states. |
| Approach: | They propose a state-independent and time-evolving Network (STN) for rumor detection based on fine-grained event state detection and segmentation. |
| Outcome: | The proposed framework can significantly improve the rumor detection accuracy in comparison with some strong baseline systems. |
End-to-End Emotion-Cause Pair Extraction based on Sliding Window Multi-Label Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to extract potential pairs of emotions ignore the fact that the cause and the emotion it triggers are inseparable. |
| Approach: | They propose two frameworks that combine multi-label learning and multi-labeled learning to extract emotion clauses . they evaluate a benchmark emotion cause corpus and find the best performance . |
| Outcome: | The proposed frameworks achieve the best performance among all compared systems on the ECPE task. |
Coupled Hierarchical Transformer for Stance-Aware Rumor Verification in Social Media Conversations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to rumor verification and stance classification fail to exploit intertask dependencies . |
| Approach: | They propose a Hierarchical Transformer model which uses BERT to obtain thread representations . they propose 'coupled' transformer modules to capture intertask interactions and a post-level attention layer to use predicted stance labels for RV. |
| Outcome: | The proposed model outperforms existing methods on two benchmark datasets. |
Generative Emotion Cause Triplet Extraction in Conversations with Commonsense Knowledge (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on ECTEC focus on Causal Emotion Entailment and Emotion-Cause Pair Extraction in Conversations. |
| Approach: | They propose to decompose the ECTEC task into multiple subtasks and solve them in a pipeline manner. |
| Outcome: | The proposed model outperforms competing systems on two benchmark datasets. |
A Joint Coreference-Aware Approach to Document-Level Target Sentiment Analysis (2024.acl-long)
Copied to clipboard
| Challenge: | Existing work on aspect-based sentiment analysis (ABSA) focuses on sentence level, document level ABSA is more practical and requires holistic document-level understanding capabilities. |
| Approach: | They propose a learning framework to jointly model the DTSA task and the coreference resolution task using ChatGPT. |
| Outcome: | The proposed framework reduces the reliance on annotated coreference information and alleviates evaluation bias caused by missing coreference targets. |
Comparative Opinion Quintuple Extraction from Product Reviews (2021.emnlp-main)
Copied to clipboard
| Challenge: | Comparative opinion mining is an important task in opinion mining. |
| Approach: | They propose a task to extract comparative opinion quintuples from product reviews . they propose supplementary annotations and construct three datasets for the task . |
| Outcome: | The proposed method outperforms baseline systems on three datasets and represents a strong benchmark for COQE. |
Language Models over Large-Scale Knowledge Base: on Capacity, Flexibility and Reasoning for New Facts (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies on LMs lack systematic studies on their structured reasoning capabilities over the infused knowledge. |
| Approach: | They investigate how LMs of different sizes can store world knowledge of different frequencies in a large-scale KB after training on the abundant world knowledge triplets. |
| Outcome: | The proposed models can store and respond to natural language queries with flexibility and reasoning abilities, but they need to be enhanced to fully realize their potential. |
Improving Multimodal Named Entity Recognition via Entity Span Detection with Unified Multimodal Transformer (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for named entity recognition ignore visual context bias . NER is a key component of many information extraction tasks . |
| Approach: | They propose to use a multimodal interaction module to generate word-aware visual representations and leverage purely text-based entity span detection as an auxiliary module to guide the final predictions. |
| Outcome: | The proposed approach achieves state-of-the-art on two benchmark datasets. |
Unified Feature and Instance Based Domain Adaptation for Aspect-Based Sentiment Analysis (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to aspect-based sentiment analysis rely on labeled data, but they lack the fine-grained labeles needed for the ABSA task. |
| Approach: | They propose a framework to perform feature adaptation and instance adaptation for the ABSA task . they learn domain-invariant feature representations by using part-of-speech features . |
| Outcome: | The proposed method improves on the state-of-the-art in two aspects of the ABSA task. |
Cross-Domain Data Augmentation with Domain-Adaptive Language Modeling for Aspect-Based Sentiment Analysis (2023.acl-long)
Copied to clipboard
| Challenge: | Cross-domain Aspect-Based Sentiment Analysis (ABSA) aims to identify aspect-sentiment pairs in sentences from a target domain. |
| Approach: | They propose a domain-adaptive language model to generate labeled data from a source domain. |
| Outcome: | The proposed approach outperforms existing methods on ABSA and Aspect Extraction tasks. |
Vision-Language Pre-Training for Multimodal Aspect-Based Sentiment Analysis (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to multimodal Aspect-Based Sentiment Analysis (MABSA) ignore crossmodalalignment and use pre-trained visual and textual models. |
| Approach: | They propose a multimodal multimodal encoder-decoder framework for MABSA that uses a unified multimodal decoder architecture for all the pretrainingand downstream tasks. |
| Outcome: | The proposed framework outperforms state-of-the-art approaches on three MABSA subtasks. |
Aspect-Category-Opinion-Sentiment Quadruple Extraction with Implicit Aspects and Opinions (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies in aspect-based sentiment analysis ignore aspects and opinions in product reviews. |
| Approach: | They propose a task to extract aspect-category-opinion-sentiment quadruples from review sentences . they construct two new datasets that contain annotations of implicit aspects and opinions . |
| Outcome: | The proposed task provides full support for aspect-based sentiment analysis with implicit aspects and opinions. |